2024-05-11T00:51:02.812 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-11T00:51:02.812 DEBUG:teuthology.run:Teuthology command: teuthology --name mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi --archive /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/snap-schedule} --owner scheduled_mchangir@teuthology --verbose -- /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/orig.config.yaml 2024-05-11T00:51:02.862 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T00:51:02.957 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038 branch: wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/snap-schedule} email: mchangir@redhat.com first_in_suite: false job_id: '7702038' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 10380055a1fc5e7d3b37b252af7045f5c2c8ba29 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 10380055a1fc5e7d3b37b252af7045f5c2c8ba29 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds sha1: 10380055a1fc5e7d3b37b252af7045f5c2c8ba29 owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3672 sha1: 10380055a1fc5e7d3b37b252af7045f5c2c8ba29 sleep_before_teardown: 0 suite: fs suite_branch: wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 10380055a1fc5e7d3b37b252af7045f5c2c8ba29 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNPzuCqcuBbnXZ3bZTAStOBBelgDv+9sQKjLwg+nqyO++1tBJi4TrMejaJOMWTNX7ndUGjaVSrhLo+v9nTXsVkQ= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOjac69GAdpZVi+7u90HWwTwSnpInKp6GsFxkoCpGY07c3VhdYd+sDz2vnYoZ+hNKkf3QRjP23ZX33X8quJrn6Q= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-11_00:39:26 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-11T00:51:02.957 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa; will attempt to use it 2024-05-11T00:51:02.958 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks 2024-05-11T00:51:02.958 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-11T00:51:02.960 INFO:teuthology.task.internal:Checking packages... 2024-05-11T00:51:02.995 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '10380055a1fc5e7d3b37b252af7045f5c2c8ba29' 2024-05-11T00:51:02.995 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-11T00:51:02.995 INFO:teuthology.packaging:ref: None 2024-05-11T00:51:02.996 INFO:teuthology.packaging:tag: None 2024-05-11T00:51:02.996 INFO:teuthology.packaging:branch: wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds 2024-05-11T00:51:02.996 INFO:teuthology.packaging:sha1: 10380055a1fc5e7d3b37b252af7045f5c2c8ba29 2024-05-11T00:51:02.996 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds 2024-05-11T00:51:03.216 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3570.g10380055 2024-05-11T00:51:03.218 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-11T00:51:03.232 INFO:teuthology.task.internal:no buildpackages task found 2024-05-11T00:51:03.233 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-11T00:51:03.261 INFO:teuthology.task.internal:Saving configuration 2024-05-11T00:51:03.282 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-11T00:51:03.294 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-11T00:51:03.319 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 00:45:54.673314', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNPzuCqcuBbnXZ3bZTAStOBBelgDv+9sQKjLwg+nqyO++1tBJi4TrMejaJOMWTNX7ndUGjaVSrhLo+v9nTXsVkQ='} 2024-05-11T00:51:03.342 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 00:45:54.674795', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOjac69GAdpZVi+7u90HWwTwSnpInKp6GsFxkoCpGY07c3VhdYd+sDz2vnYoZ+hNKkf3QRjP23ZX33X8quJrn6Q='} 2024-05-11T00:51:03.342 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-11T00:51:03.354 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-11T00:51:03.354 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-11T00:51:03.354 INFO:teuthology.run_tasks:Running task console_log... 2024-05-11T00:51:03.447 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa392f4af70>, signals=[15]) 2024-05-11T00:51:03.448 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-11T00:51:03.459 INFO:teuthology.task.internal:Opening connections... 2024-05-11T00:51:03.460 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T00:51:03.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:51:03.543 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T00:51:03.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:51:03.618 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-11T00:51:03.629 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-05-11T00:51:03.648 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-05-11T00:51:03.648 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T00:51:03.707 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/" 2024-05-11T00:51:03.708 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T00:51:03.708 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T00:51:03.708 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T00:51:03.708 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-05-11T00:51:03.732 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-05-11T00:51:03.750 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-05-11T00:51:03.750 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-05-11T00:51:03.807 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-05-11T00:51:03.807 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-05-11T00:51:03.807 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-05-11T00:51:03.807 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-05-11T00:51:03.807 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-05-11T00:51:03.807 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-05-11T00:51:03.807 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T00:51:03.808 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-05-11T00:51:03.808 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-05-11T00:51:03.808 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T00:51:03.808 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-05-11T00:51:03.808 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T00:51:03.808 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T00:51:03.808 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T00:51:03.808 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-05-11T00:51:03.832 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-11T00:51:03.842 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-11T00:51:03.853 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-11T00:51:03.853 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-05-11T00:51:03.855 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-05-11T00:51:03.871 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-11T00:51:03.912 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-11T00:51:03.913 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-05-11T00:51:03.915 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-05-11T00:51:03.942 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-11T00:51:04.004 INFO:teuthology.run_tasks:Running task kernel... 2024-05-11T00:51:04.021 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-11T00:51:04.021 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-11T00:51:04.022 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T00:51:04.022 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T00:51:04.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T00:51:04.039 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-05-11T00:51:04.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T00:51:04.040 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-05-11T00:51:04.098 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-437.el9.x86_64 2024-05-11T00:51:04.098 INFO:teuthology.task.kernel:Running kernel on smithi119: 5.14.0-437.el9.x86_64 2024-05-11T00:51:04.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum install -y kernel 2024-05-11T00:51:04.100 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-437.el9.x86_64 2024-05-11T00:51:04.101 INFO:teuthology.task.kernel:Running kernel on smithi052: 5.14.0-437.el9.x86_64 2024-05-11T00:51:04.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum install -y kernel 2024-05-11T00:51:07.274 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.1 MB 00:01 2024-05-11T00:51:09.966 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.1 MB 00:04 2024-05-11T00:51:12.675 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-05-11T00:51:18.892 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-05-11T00:51:20.240 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 1.6 MB/s | 19 MB 00:11 2024-05-11T00:51:20.501 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-05-11T00:51:24.416 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-11T00:51:25.825 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 16 kB 00:00 2024-05-11T00:51:26.563 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 21 MB 00:05 2024-05-11T00:51:27.621 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-11T00:51:31.685 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-05-11T00:51:32.968 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-11T00:51:33.641 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-11T00:51:33.641 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-11T00:51:33.731 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-11T00:51:33.736 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout:Install 4 Packages 2024-05-11T00:51:33.737 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:51:33.738 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 78 M 2024-05-11T00:51:33.738 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 123 M 2024-05-11T00:51:33.738 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-11T00:51:34.111 INFO:teuthology.orchestra.run.smithi119.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.5 MB/s | 484 kB 00:00 2024-05-11T00:51:35.083 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-11T00:51:35.084 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-11T00:51:35.189 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-05-11T00:51:35.208 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-05-11T00:51:35.208 INFO:teuthology.orchestra.run.smithi052.stdout: Package Architecture Version Repository Size 2024-05-11T00:51:35.208 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-05-11T00:51:35.208 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-05-11T00:51:35.208 INFO:teuthology.orchestra.run.smithi052.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-11T00:51:35.209 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-05-11T00:51:35.209 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-11T00:51:35.209 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-11T00:51:35.209 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-11T00:51:35.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:51:35.209 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-05-11T00:51:35.209 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-05-11T00:51:35.209 INFO:teuthology.orchestra.run.smithi052.stdout:Install 4 Packages 2024-05-11T00:51:35.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:51:35.210 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 78 M 2024-05-11T00:51:35.210 INFO:teuthology.orchestra.run.smithi052.stdout:Installed size: 123 M 2024-05-11T00:51:35.210 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-05-11T00:51:35.404 INFO:teuthology.orchestra.run.smithi119.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 9.7 MB/s | 15 MB 00:01 2024-05-11T00:51:36.220 INFO:teuthology.orchestra.run.smithi119.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 13 MB/s | 28 MB 00:02 2024-05-11T00:51:36.436 INFO:teuthology.orchestra.run.smithi052.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 416 kB/s | 484 kB 00:01 2024-05-11T00:51:36.687 INFO:teuthology.orchestra.run.smithi119.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 12 MB/s | 34 MB 00:02 2024-05-11T00:51:36.687 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-11T00:51:36.688 INFO:teuthology.orchestra.run.smithi119.stdout:Total 26 MB/s | 78 MB 00:02 2024-05-11T00:51:37.367 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-11T00:51:37.387 INFO:teuthology.orchestra.run.smithi052.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 7.3 MB/s | 15 MB 00:02 2024-05-11T00:51:37.707 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-11T00:51:37.707 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-11T00:51:38.037 INFO:teuthology.orchestra.run.smithi052.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 12 MB/s | 34 MB 00:02 2024-05-11T00:51:38.601 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-11T00:51:38.601 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-11T00:51:39.395 INFO:teuthology.orchestra.run.smithi052.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 9.5 MB/s | 28 MB 00:02 2024-05-11T00:51:39.396 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-05-11T00:51:39.396 INFO:teuthology.orchestra.run.smithi052.stdout:Total 19 MB/s | 78 MB 00:04 2024-05-11T00:51:40.104 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-05-11T00:51:40.506 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-05-11T00:51:40.506 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-05-11T00:51:40.994 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-11T00:51:41.451 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-05-11T00:51:41.451 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-05-11T00:51:41.913 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-11T00:51:42.029 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T00:51:43.987 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-05-11T00:51:44.620 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-11T00:51:44.698 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T00:51:44.704 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T00:51:44.834 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T00:51:47.490 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T00:51:47.625 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T00:51:49.562 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T00:51:49.679 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:51:52.706 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T00:51:52.752 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:51:54.423 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:51:57.846 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:52:23.896 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:52:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:52:24.230 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:52:24.230 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-11T00:52:24.230 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T00:52:24.230 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T00:52:25.040 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:52:25.040 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:52:25.040 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-11T00:52:25.040 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-11T00:52:25.040 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-11T00:52:25.041 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-11T00:52:25.041 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-11T00:52:25.041 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:52:25.041 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-11T00:52:25.395 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T00:52:25.468 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T00:52:25.468 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T00:52:25.468 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-11T00:52:25.468 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-11T00:52:25.468 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-11T00:52:25.469 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-11T00:52:25.469 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=None, version=distro) 2024-05-11T00:52:25.469 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T00:52:25.517 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T00:52:25.518 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T00:52:25.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2024-05-11T00:52:25.548 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.06 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 68.el9 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Group : Unspecified 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 8264254 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T00:52:25.549 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T00:52:25.550 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : builder@centos.org 2024-05-11T00:52:25.550 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2024-05-11T00:52:25.550 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T00:52:25.550 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2024-05-11T00:52:25.550 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2024-05-11T00:52:25.550 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:52:25.550 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T00:52:25.554 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T00:52:25.554 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T00:52:25.554 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2024-05-11T00:52:25.554 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:52:25.554 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T00:52:25.555 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T00:52:25.555 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 5.14.0-444.el9.x86_64 2024-05-11T00:52:25.555 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T00:52:26.180 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-05-11T00:52:27.534 INFO:teuthology.orchestra.run.smithi119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T00:52:27.563 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-05-11T00:52:27.565 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-05-11T00:52:27.580 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.SOOUj20QjV 2024-05-11T00:52:27.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SOOUj20QjV 2024-05-11T00:52:27.645 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.SOOUj20QjV 2024-05-11T00:52:27.791 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.SOOUj20QjV is 6KB 2024-05-11T00:52:27.840 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.SOOUj20QjV 2024-05-11T00:52:27.854 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T00:52:27.922 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T00:52:27.923 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T00:52:27.923 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T00:52:27.923 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-11T00:52:27.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-11T00:52:28.058 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-05-11T00:52:28.763 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:52:28.804 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:52:29.149 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:52:29.149 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-11T00:52:29.149 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T00:52:29.149 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T00:52:29.970 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T00:52:29.970 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:52:29.970 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-05-11T00:52:29.970 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-11T00:52:29.970 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-11T00:52:29.971 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-11T00:52:29.971 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-11T00:52:29.971 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:52:29.971 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-05-11T00:52:30.304 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T00:52:30.377 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T00:52:30.377 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T00:52:30.377 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-11T00:52:30.377 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-11T00:52:30.377 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-11T00:52:30.377 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-11T00:52:30.377 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi052.front.sepia.ceph.com, path=None, version=distro) 2024-05-11T00:52:30.378 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T00:52:30.428 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T00:52:30.428 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T00:52:30.429 DEBUG:teuthology.orchestra.run.smithi052:> sudo rpm -qi grub2-tools 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:Name : grub2-tools 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:Epoch : 1 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:Version : 2.06 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:Release : 68.el9 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:Architecture: x86_64 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:Group : Unspecified 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:Size : 8264254 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:License : GPLv3+ 2024-05-11T00:52:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:Packager : builder@centos.org 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:Vendor : CentOS 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:Summary : Support tools for GRUB. 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:Description : 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:hardware devices. 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:52:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T00:52:30.469 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T00:52:30.469 INFO:teuthology.task.kernel:Updating grub on smithi052 to boot 5.14.0-444.el9.x86_64 2024-05-11T00:52:30.469 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T00:52:31.023 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-05-11T00:52:32.388 INFO:teuthology.orchestra.run.smithi052.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T00:52:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-05-11T00:52:32.416 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-05-11T00:52:32.454 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.79MDwCMLSB 2024-05-11T00:52:32.454 DEBUG:teuthology.orchestra.run.smithi052:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.79MDwCMLSB 2024-05-11T00:52:32.520 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.79MDwCMLSB 2024-05-11T00:52:32.658 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.79MDwCMLSB is 6KB 2024-05-11T00:52:32.707 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.79MDwCMLSB 2024-05-11T00:52:32.722 DEBUG:teuthology.orchestra.run.smithi052:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T00:52:32.784 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T00:52:32.784 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T00:52:32.784 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T00:52:32.784 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-11T00:52:32.785 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-11T00:52:32.941 DEBUG:teuthology.orchestra.run.smithi052:> sudo shutdown -r now 2024-05-11T00:52:58.086 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-11T00:52:58.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-11T00:52:58.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:53:02.946 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-11T00:53:02.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-11T00:53:02.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:53:16.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-05-11T00:53:21.314 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.52 2024-05-11T00:53:25.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-11T00:53:25.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:53:28.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-05-11T00:53:30.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-11T00:53:30.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:53:33.442 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.52 2024-05-11T00:53:40.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-11T00:53:40.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:53:45.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-11T00:53:45.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:53:58.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-05-11T00:54:05.286 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-05-11T00:54:05.434 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-11T00:54:05.435 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-11T00:54:05.435 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-05-11T00:54:05.455 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-444.el9.x86_64 2024-05-11T00:54:05.455 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-11T00:54:05.455 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T00:54:05.455 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T00:54:06.457 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-11T00:54:06.457 DEBUG:teuthology.orchestra.run.smithi052:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-11T00:54:06.582 INFO:teuthology.orchestra.run.smithi052.stdout:ttyS1 2024-05-11T00:54:06.632 DEBUG:teuthology.parallel:result is None 2024-05-11T00:54:13.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-11T00:54:13.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:54:14.307 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-05-11T00:54:14.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-11T00:54:14.767 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-11T00:54:14.767 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-05-11T00:54:14.781 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-444.el9.x86_64 2024-05-11T00:54:14.781 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-11T00:54:14.782 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T00:54:14.782 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T00:54:15.782 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-11T00:54:15.783 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-11T00:54:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2024-05-11T00:54:15.928 DEBUG:teuthology.parallel:result is None 2024-05-11T00:54:15.928 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-11T00:54:15.939 INFO:teuthology.task.internal:Creating test directory... 2024-05-11T00:54:15.939 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T00:54:15.943 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T00:54:15.969 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-11T00:54:16.013 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-11T00:54:16.050 INFO:teuthology.task.internal:Creating archive directory... 2024-05-11T00:54:16.050 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T00:54:16.053 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T00:54:16.087 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-11T00:54:16.115 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-11T00:54:16.116 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T00:54:16.119 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T00:54:16.146 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T00:54:16.157 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T00:54:16.177 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T00:54:16.195 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T00:54:16.197 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-11T00:54:16.208 INFO:teuthology.task.internal:Configuring sudo... 2024-05-11T00:54:16.209 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T00:54:16.211 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T00:54:16.255 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-11T00:54:16.264 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-11T00:54:16.264 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T00:54:16.278 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T00:54:16.314 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T00:54:16.375 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T00:54:16.435 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T00:54:16.519 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T00:54:16.560 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T00:54:16.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T00:54:16.627 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T00:54:16.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T00:54:16.781 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T00:54:16.819 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T00:54:16.866 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T00:54:16.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T00:54:16.934 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-05-11T00:54:16.937 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-05-11T00:54:16.984 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T00:54:17.014 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T00:54:17.432 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-11T00:54:17.439 INFO:teuthology.task.internal:Starting timer... 2024-05-11T00:54:17.440 INFO:teuthology.run_tasks:Running task pcp... 2024-05-11T00:54:17.501 INFO:teuthology.run_tasks:Running task selinux... 2024-05-11T00:54:17.509 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-11T00:54:17.509 DEBUG:teuthology.orchestra.run.smithi052:> sudo service auditd rotate 2024-05-11T00:54:17.566 INFO:teuthology.orchestra.run.smithi052.stdout:Rotating logs: 2024-05-11T00:54:17.568 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-05-11T00:54:17.617 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-05-11T00:54:17.619 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-11T00:54:17.619 DEBUG:teuthology.orchestra.run.smithi052:> /usr/sbin/getenforce 2024-05-11T00:54:17.642 INFO:teuthology.orchestra.run.smithi052.stdout:Permissive 2024-05-11T00:54:17.642 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-05-11T00:54:17.662 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-05-11T00:54:17.662 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi052.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi119.front.sepia.ceph.com': 'permissive'} 2024-05-11T00:54:17.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T00:54:17.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T00:54:17.712 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T00:54:17.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T00:54:17.737 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-11T00:54:17.737 DEBUG:teuthology.orchestra.run.smithi052:> sudo /usr/sbin/setenforce permissive 2024-05-11T00:54:17.782 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-05-11T00:54:17.807 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-11T00:54:17.816 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-11T00:54:17.822 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-11T00:54:17.959 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-11T00:54:17.992 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-11T00:54:17.994 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi052.front.sepia.ceph.com,smithi119.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-11T00:58:22.770 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com')] 2024-05-11T00:58:22.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-11T00:58:22.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:58:22.854 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-05-11T00:58:22.936 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-11T00:58:22.936 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-11T00:58:22.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T00:58:23.014 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-05-11T00:58:23.096 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-11T00:58:23.096 INFO:teuthology.run_tasks:Running task clock... 2024-05-11T00:58:23.108 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-11T00:58:23.108 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T00:58:23.108 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T00:58:23.111 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T00:58:23.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T00:58:23.142 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T00:58:23.158 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T00:58:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T00:58:23.182 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T00:58:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: ntpd: command not found 2024-05-11T00:58:23.196 INFO:teuthology.orchestra.run.smithi052.stdout:506 Cannot talk to daemon 2024-05-11T00:58:23.207 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-05-11T00:58:23.211 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T00:58:23.218 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-05-11T00:58:23.225 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T00:58:23.233 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T00:58:23.247 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T00:58:23.282 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-05-11T00:58:23.285 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T00:58:23.285 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-05-11T00:58:23.285 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T00:58:23.285 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T00:58:23.285 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T00:58:23.285 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T00:58:23.299 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-05-11T00:58:23.302 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T00:58:23.302 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-05-11T00:58:23.302 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T00:58:23.303 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T00:58:23.303 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T00:58:23.303 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T00:58:23.304 INFO:teuthology.run_tasks:Running task install... 2024-05-11T00:58:23.313 DEBUG:teuthology.task.install:project ceph 2024-05-11T00:58:23.313 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29'}} 2024-05-11T00:58:23.313 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29'} 2024-05-11T00:58:23.313 INFO:teuthology.task.install:Using flavor: default 2024-05-11T00:58:23.349 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-11T00:58:23.349 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-11T00:58:23.350 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29', 'tag': None, 'wait_for_package': False} 2024-05-11T00:58:23.350 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=10380055a1fc5e7d3b37b252af7045f5c2c8ba29 2024-05-11T00:58:23.352 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29', 'tag': None, 'wait_for_package': False} 2024-05-11T00:58:23.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=10380055a1fc5e7d3b37b252af7045f5c2c8ba29 2024-05-11T00:58:23.495 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/centos/9/flavors/default/ 2024-05-11T00:58:23.495 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3570.g10380055 2024-05-11T00:58:23.590 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/centos/9/flavors/default/ 2024-05-11T00:58:23.590 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3570.g10380055 2024-05-11T00:58:23.610 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T00:58:23.610 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T00:58:23.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T00:58:23.635 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T00:58:23.635 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T00:58:23.700 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T00:58:23.700 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T00:58:23.700 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T00:58:23.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T00:58:23.729 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T00:58:23.729 DEBUG:teuthology.orchestra.run.smithi052:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/10380055a1fc5e7d3b37b252af7045f5c2c8ba29/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T00:58:23.776 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T00:58:23.796 DEBUG:teuthology.orchestra.run.smithi052:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T00:58:23.841 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-05-11T00:58:23.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-05-11T00:58:23.874 DEBUG:teuthology.orchestra.run.smithi052:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T00:58:23.939 INFO:teuthology.orchestra.run.smithi052.stdout:check_obsoletes = 1 2024-05-11T00:58:23.940 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum clean all 2024-05-11T00:58:24.068 INFO:teuthology.orchestra.run.smithi119.stdout:54 files removed 2024-05-11T00:58:24.095 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T00:58:24.188 INFO:teuthology.orchestra.run.smithi052.stdout:54 files removed 2024-05-11T00:58:24.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T00:58:24.794 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 196 kB/s | 84 kB 00:00 2024-05-11T00:58:24.832 INFO:teuthology.orchestra.run.smithi052.stdout:ceph packages for x86_64 254 kB/s | 84 kB 00:00 2024-05-11T00:58:25.283 INFO:teuthology.orchestra.run.smithi052.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-11T00:58:25.295 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-05-11T00:58:25.692 INFO:teuthology.orchestra.run.smithi052.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-05-11T00:58:26.237 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 2.1 kB/s | 1.9 kB 00:00 2024-05-11T00:58:26.267 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-11T00:58:28.020 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-11T00:58:28.068 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-05-11T00:58:30.113 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-05-11T00:58:32.245 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.2 MB 00:00 2024-05-11T00:58:33.838 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-05-11T00:58:34.588 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.2 MB 00:00 2024-05-11T00:58:36.473 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-05-11T00:58:37.610 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-05-11T00:58:41.892 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 21 MB 00:07 2024-05-11T00:58:42.958 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 7.2 kB/s | 1.7 kB 00:00 2024-05-11T00:58:45.251 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T00:58:45.251 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T00:58:45.262 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T00:58:45.263 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T00:58:45.263 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T00:58:45.264 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T00:58:45.266 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T00:58:45.268 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T00:58:45.269 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T00:58:45.329 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-11T00:58:45.361 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-05-11T00:58:45.361 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-05-11T00:58:45.362 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-05-11T00:58:45.362 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-11T00:58:45.362 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T00:58:45.362 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.0.0-3570.g10380055.el9 ceph 6.6 k 2024-05-11T00:58:45.362 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.0.0-3570.g10380055.el9 ceph 5.4 M 2024-05-11T00:58:45.362 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.0.0-3570.g10380055.el9 ceph 890 k 2024-05-11T00:58:45.362 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3570.g10380055.el9 ceph 145 k 2024-05-11T00:58:45.363 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.0.0-3570.g10380055.el9 ceph 1.5 M 2024-05-11T00:58:45.363 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 142 k 2024-05-11T00:58:45.363 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 3.5 M 2024-05-11T00:58:45.363 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 7.4 M 2024-05-11T00:58:45.363 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 49 k 2024-05-11T00:58:45.363 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.0.0-3570.g10380055.el9 ceph 11 M 2024-05-11T00:58:45.364 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.0.0-3570.g10380055.el9 ceph 49 M 2024-05-11T00:58:45.364 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 273 k 2024-05-11T00:58:45.364 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 763 k 2024-05-11T00:58:45.364 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:19.0.0-3570.g10380055.el9 ceph 224 k 2024-05-11T00:58:45.364 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell x86_64 2:19.0.0-3570.g10380055.el9 ceph 27 k 2024-05-11T00:58:45.364 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 18 k 2024-05-11T00:58:45.365 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T00:58:45.365 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T00:58:45.365 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T00:58:45.365 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.0.0-3570.g10380055.el9 ceph 32 k 2024-05-11T00:58:45.365 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.0.0-3570.g10380055.el9 ceph 747 k 2024-05-11T00:58:45.365 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.0.0-3570.g10380055.el9 ceph 127 k 2024-05-11T00:58:45.366 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T00:58:45.366 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T00:58:45.366 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T00:58:45.366 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T00:58:45.366 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T00:58:45.366 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T00:58:45.367 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.0.0-3570.g10380055.el9 ceph 161 k 2024-05-11T00:58:45.367 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.0.0-3570.g10380055.el9 ceph 321 k 2024-05-11T00:58:45.367 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.0.0-3570.g10380055.el9 ceph 300 k 2024-05-11T00:58:45.367 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.0.0-3570.g10380055.el9 ceph 99 k 2024-05-11T00:58:45.367 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.0.0-3570.g10380055.el9 ceph 85 k 2024-05-11T00:58:45.367 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.0.0-3570.g10380055.el9 ceph 3.1 M 2024-05-11T00:58:45.367 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.0.0-3570.g10380055.el9 ceph 170 k 2024-05-11T00:58:45.368 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T00:58:45.368 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-05-11T00:58:45.368 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T00:58:45.368 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T00:58:45.368 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T00:58:45.368 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T00:58:45.369 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.0.0-3570.g10380055.el9 ceph 3.4 M 2024-05-11T00:58:45.369 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.0.0-3570.g10380055.el9 ceph 3.1 M 2024-05-11T00:58:45.369 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T00:58:45.369 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T00:58:45.369 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-11T00:58:45.369 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T00:58:45.369 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T00:58:45.370 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T00:58:45.370 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.0.0-3570.g10380055.el9 ceph 21 M 2024-05-11T00:58:45.370 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 34 k 2024-05-11T00:58:45.370 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.0.0-3570.g10380055.el9 ceph 2.4 M 2024-05-11T00:58:45.370 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 274 k 2024-05-11T00:58:45.370 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.0.0-3570.g10380055.el9 ceph 4.7 M 2024-05-11T00:58:45.370 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.0.0-3570.g10380055.el9 ceph 17 M 2024-05-11T00:58:45.371 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 16 k 2024-05-11T00:58:45.371 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.0.0-3570.g10380055.el9 ceph 25 k 2024-05-11T00:58:45.371 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T00:58:45.371 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T00:58:45.371 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T00:58:45.371 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T00:58:45.371 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T00:58:45.372 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T00:58:45.372 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T00:58:45.372 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T00:58:45.372 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.0.0-3570.g10380055.el9 ceph 164 k 2024-05-11T00:58:45.372 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T00:58:45.372 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T00:58:45.372 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T00:58:45.373 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T00:58:45.373 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T00:58:45.373 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.0.0-3570.g10380055.el9 ceph 501 k 2024-05-11T00:58:45.373 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T00:58:45.373 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.0.0-3570.g10380055.el9 ceph 5.4 M 2024-05-11T00:58:45.373 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T00:58:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T00:58:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T00:58:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T00:58:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T00:58:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T00:58:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T00:58:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T00:58:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T00:58:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T00:58:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T00:58:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T00:58:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T00:58:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T00:58:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T00:58:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T00:58:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T00:58:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.0.0-3570.g10380055.el9 ceph 45 k 2024-05-11T00:58:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.0.0-3570.g10380055.el9 ceph 135 k 2024-05-11T00:58:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T00:58:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T00:58:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T00:58:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T00:58:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T00:58:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T00:58:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T00:58:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T00:58:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T00:58:45.381 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-05-11T00:58:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T00:58:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:58:45.381 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-11T00:58:45.381 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-05-11T00:58:45.381 INFO:teuthology.orchestra.run.smithi119.stdout:Install 148 Packages 2024-05-11T00:58:45.381 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 8 Packages 2024-05-11T00:58:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:58:45.382 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 226 M 2024-05-11T00:58:45.382 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-11T00:58:46.035 INFO:teuthology.orchestra.run.smithi119.stdout:(1/156): ceph-19.0.0-3570.g10380055.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-05-11T00:58:46.277 INFO:teuthology.orchestra.run.smithi119.stdout:(2/156): ceph-fuse-19.0.0-3570.g10380055.el9.x8 3.6 MB/s | 890 kB 00:00 2024-05-11T00:58:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:(3/156): ceph-immutable-object-cache-19.0.0-357 600 kB/s | 145 kB 00:00 2024-05-11T00:58:46.952 INFO:teuthology.orchestra.run.smithi119.stdout:(4/156): ceph-mds-19.0.0-3570.g10380055.el9.x86 5.5 MB/s | 2.4 MB 00:00 2024-05-11T00:58:47.181 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-11T00:58:47.186 INFO:teuthology.orchestra.run.smithi119.stdout:(5/156): ceph-common-19.0.0-3570.g10380055.el9. 16 MB/s | 21 MB 00:01 2024-05-11T00:58:47.228 INFO:teuthology.orchestra.run.smithi119.stdout:(6/156): ceph-mgr-19.0.0-3570.g10380055.el9.x86 5.5 MB/s | 1.5 MB 00:00 2024-05-11T00:58:47.836 INFO:teuthology.orchestra.run.smithi119.stdout:(7/156): ceph-mon-19.0.0-3570.g10380055.el9.x86 7.2 MB/s | 4.7 MB 00:00 2024-05-11T00:58:48.636 INFO:teuthology.orchestra.run.smithi119.stdout:(8/156): ceph-radosgw-19.0.0-3570.g10380055.el9 13 MB/s | 11 MB 00:00 2024-05-11T00:58:48.803 INFO:teuthology.orchestra.run.smithi119.stdout:(9/156): ceph-osd-19.0.0-3570.g10380055.el9.x86 11 MB/s | 17 MB 00:01 2024-05-11T00:58:48.820 INFO:teuthology.orchestra.run.smithi119.stdout:(10/156): ceph-selinux-19.0.0-3570.g10380055.el 137 kB/s | 25 kB 00:00 2024-05-11T00:58:48.912 INFO:teuthology.orchestra.run.smithi119.stdout:(11/156): cephfs-mirror-19.0.0-3570.g10380055.e 2.4 MB/s | 224 kB 00:00 2024-05-11T00:58:48.962 INFO:teuthology.orchestra.run.smithi119.stdout:(12/156): cephfs-shell-19.0.0-3570.g10380055.el 541 kB/s | 27 kB 00:00 2024-05-11T00:58:49.171 INFO:teuthology.orchestra.run.smithi119.stdout:(13/156): libcephfs-devel-19.0.0-3570.g10380055 152 kB/s | 32 kB 00:00 2024-05-11T00:58:49.288 INFO:teuthology.orchestra.run.smithi119.stdout:(14/156): libcephfs2-19.0.0-3570.g10380055.el9. 6.3 MB/s | 747 kB 00:00 2024-05-11T00:58:49.355 INFO:teuthology.orchestra.run.smithi119.stdout:(15/156): libcephsqlite-19.0.0-3570.g10380055.e 2.4 MB/s | 164 kB 00:00 2024-05-11T00:58:49.414 INFO:teuthology.orchestra.run.smithi119.stdout:(16/156): librados-devel-19.0.0-3570.g10380055. 2.1 MB/s | 127 kB 00:00 2024-05-11T00:58:49.583 INFO:teuthology.orchestra.run.smithi052.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T00:58:49.584 INFO:teuthology.orchestra.run.smithi052.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T00:58:49.595 INFO:teuthology.orchestra.run.smithi052.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T00:58:49.596 INFO:teuthology.orchestra.run.smithi052.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T00:58:49.597 INFO:teuthology.orchestra.run.smithi052.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T00:58:49.597 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T00:58:49.599 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T00:58:49.601 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T00:58:49.602 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T00:58:49.639 INFO:teuthology.orchestra.run.smithi119.stdout:(17/156): libradosstriper1-19.0.0-3570.g1038005 2.2 MB/s | 501 kB 00:00 2024-05-11T00:58:49.667 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-05-11T00:58:49.689 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-05-11T00:58:49.689 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph x86_64 2:19.0.0-3570.g10380055.el9 ceph 6.6 k 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base x86_64 2:19.0.0-3570.g10380055.el9 ceph 5.4 M 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse x86_64 2:19.0.0-3570.g10380055.el9 ceph 890 k 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3570.g10380055.el9 ceph 145 k 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr x86_64 2:19.0.0-3570.g10380055.el9 ceph 1.5 M 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 142 k 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 3.5 M 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 7.4 M 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 49 k 2024-05-11T00:58:49.690 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw x86_64 2:19.0.0-3570.g10380055.el9 ceph 11 M 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test x86_64 2:19.0.0-3570.g10380055.el9 ceph 49 M 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 273 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 763 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror x86_64 2:19.0.0-3570.g10380055.el9 ceph 224 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell x86_64 2:19.0.0-3570.g10380055.el9 ceph 27 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 18 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel x86_64 2:19.0.0-3570.g10380055.el9 ceph 32 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2 x86_64 2:19.0.0-3570.g10380055.el9 ceph 747 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel x86_64 2:19.0.0-3570.g10380055.el9 ceph 127 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T00:58:49.691 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs x86_64 2:19.0.0-3570.g10380055.el9 ceph 161 k 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados x86_64 2:19.0.0-3570.g10380055.el9 ceph 321 k 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd x86_64 2:19.0.0-3570.g10380055.el9 ceph 300 k 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw x86_64 2:19.0.0-3570.g10380055.el9 ceph 99 k 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse x86_64 2:19.0.0-3570.g10380055.el9 ceph 85 k 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror x86_64 2:19.0.0-3570.g10380055.el9 ceph 3.1 M 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd x86_64 2:19.0.0-3570.g10380055.el9 ceph 170 k 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T00:58:49.692 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T00:58:49.693 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T00:58:49.693 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T00:58:49.693 INFO:teuthology.orchestra.run.smithi052.stdout: librados2 x86_64 2:19.0.0-3570.g10380055.el9 ceph 3.4 M 2024-05-11T00:58:49.693 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1 x86_64 2:19.0.0-3570.g10380055.el9 ceph 3.1 M 2024-05-11T00:58:49.693 INFO:teuthology.orchestra.run.smithi052.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T00:58:49.693 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T00:58:49.693 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-05-11T00:58:49.694 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T00:58:49.694 INFO:teuthology.orchestra.run.smithi052.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T00:58:49.695 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T00:58:49.695 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common x86_64 2:19.0.0-3570.g10380055.el9 ceph 21 M 2024-05-11T00:58:49.695 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 34 k 2024-05-11T00:58:49.695 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds x86_64 2:19.0.0-3570.g10380055.el9 ceph 2.4 M 2024-05-11T00:58:49.695 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 274 k 2024-05-11T00:58:49.695 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon x86_64 2:19.0.0-3570.g10380055.el9 ceph 4.7 M 2024-05-11T00:58:49.695 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd x86_64 2:19.0.0-3570.g10380055.el9 ceph 17 M 2024-05-11T00:58:49.695 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3570.g10380055.el9 ceph-noarch 16 k 2024-05-11T00:58:49.695 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux x86_64 2:19.0.0-3570.g10380055.el9 ceph 25 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite x86_64 2:19.0.0-3570.g10380055.el9 ceph 164 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T00:58:49.696 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 x86_64 2:19.0.0-3570.g10380055.el9 ceph 501 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2 x86_64 2:19.0.0-3570.g10380055.el9 ceph 5.4 M 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T00:58:49.697 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T00:58:49.698 INFO:teuthology.orchestra.run.smithi052.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T00:58:49.698 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T00:58:49.698 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T00:58:49.698 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T00:58:49.698 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T00:58:49.698 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T00:58:49.698 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T00:58:49.698 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T00:58:49.698 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse x86_64 2:19.0.0-3570.g10380055.el9 ceph 45 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common x86_64 2:19.0.0-3570.g10380055.el9 ceph 135 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T00:58:49.699 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T00:58:49.700 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T00:58:49.700 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T00:58:49.700 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T00:58:49.700 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T00:58:49.700 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T00:58:49.700 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T00:58:49.700 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T00:58:49.700 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T00:58:49.700 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T00:58:49.701 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T00:58:49.701 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T00:58:49.701 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T00:58:49.701 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T00:58:49.701 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T00:58:49.701 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T00:58:49.701 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T00:58:49.701 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T00:58:49.702 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T00:58:49.703 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T00:58:49.704 INFO:teuthology.orchestra.run.smithi052.stdout:Installing weak dependencies: 2024-05-11T00:58:49.705 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T00:58:49.705 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:58:49.705 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-05-11T00:58:49.705 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-05-11T00:58:49.705 INFO:teuthology.orchestra.run.smithi052.stdout:Install 148 Packages 2024-05-11T00:58:49.705 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 8 Packages 2024-05-11T00:58:49.705 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:58:49.706 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 226 M 2024-05-11T00:58:49.706 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-05-11T00:58:50.081 INFO:teuthology.orchestra.run.smithi119.stdout:(18/156): librgw2-19.0.0-3570.g10380055.el9.x86 12 MB/s | 5.4 MB 00:00 2024-05-11T00:58:50.139 INFO:teuthology.orchestra.run.smithi119.stdout:(19/156): python3-ceph-argparse-19.0.0-3570.g10 776 kB/s | 45 kB 00:00 2024-05-11T00:58:50.298 INFO:teuthology.orchestra.run.smithi119.stdout:(20/156): python3-ceph-common-19.0.0-3570.g1038 855 kB/s | 135 kB 00:00 2024-05-11T00:58:50.349 INFO:teuthology.orchestra.run.smithi119.stdout:(21/156): python3-cephfs-19.0.0-3570.g10380055. 3.2 MB/s | 161 kB 00:00 2024-05-11T00:58:50.441 INFO:teuthology.orchestra.run.smithi119.stdout:(22/156): python3-rados-19.0.0-3570.g10380055.e 3.4 MB/s | 321 kB 00:00 2024-05-11T00:58:50.474 INFO:teuthology.orchestra.run.smithi052.stdout:(1/156): ceph-19.0.0-3570.g10380055.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-05-11T00:58:50.608 INFO:teuthology.orchestra.run.smithi119.stdout:(23/156): ceph-base-19.0.0-3570.g10380055.el9.x 1.1 MB/s | 5.4 MB 00:04 2024-05-11T00:58:50.641 INFO:teuthology.orchestra.run.smithi119.stdout:(24/156): python3-rbd-19.0.0-3570.g10380055.el9 1.5 MB/s | 300 kB 00:00 2024-05-11T00:58:50.708 INFO:teuthology.orchestra.run.smithi119.stdout:(25/156): rbd-fuse-19.0.0-3570.g10380055.el9.x8 1.3 MB/s | 85 kB 00:00 2024-05-11T00:58:50.816 INFO:teuthology.orchestra.run.smithi052.stdout:(2/156): ceph-fuse-19.0.0-3570.g10380055.el9.x8 2.5 MB/s | 890 kB 00:00 2024-05-11T00:58:50.991 INFO:teuthology.orchestra.run.smithi052.stdout:(3/156): ceph-immutable-object-cache-19.0.0-357 827 kB/s | 145 kB 00:00 2024-05-11T00:58:51.092 INFO:teuthology.orchestra.run.smithi119.stdout:(26/156): rbd-mirror-19.0.0-3570.g10380055.el9. 8.1 MB/s | 3.1 MB 00:00 2024-05-11T00:58:51.117 INFO:teuthology.orchestra.run.smithi119.stdout:(27/156): python3-rgw-19.0.0-3570.g10380055.el9 195 kB/s | 99 kB 00:00 2024-05-11T00:58:51.168 INFO:teuthology.orchestra.run.smithi119.stdout:(28/156): rbd-nbd-19.0.0-3570.g10380055.el9.x86 2.2 MB/s | 170 kB 00:00 2024-05-11T00:58:51.210 INFO:teuthology.orchestra.run.smithi119.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3570.g 372 kB/s | 34 kB 00:00 2024-05-11T00:58:51.243 INFO:teuthology.orchestra.run.smithi119.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3570.g1038005 1.8 MB/s | 142 kB 00:00 2024-05-11T00:58:51.583 INFO:teuthology.orchestra.run.smithi052.stdout:(4/156): ceph-mds-19.0.0-3570.g10380055.el9.x86 4.0 MB/s | 2.4 MB 00:00 2024-05-11T00:58:52.093 INFO:teuthology.orchestra.run.smithi119.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 8.7 MB/s | 7.4 MB 00:00 2024-05-11T00:58:52.125 INFO:teuthology.orchestra.run.smithi052.stdout:(5/156): ceph-mgr-19.0.0-3570.g10380055.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-05-11T00:58:52.194 INFO:teuthology.orchestra.run.smithi119.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3570.g10 2.7 MB/s | 274 kB 00:00 2024-05-11T00:58:52.286 INFO:teuthology.orchestra.run.smithi119.stdout:(33/156): ceph-mgr-rook-19.0.0-3570.g10380055.e 538 kB/s | 49 kB 00:00 2024-05-11T00:58:52.386 INFO:teuthology.orchestra.run.smithi119.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3570.g1 164 kB/s | 16 kB 00:00 2024-05-11T00:58:52.909 INFO:teuthology.orchestra.run.smithi052.stdout:(6/156): ceph-base-19.0.0-3570.g10380055.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-05-11T00:58:53.286 INFO:teuthology.orchestra.run.smithi119.stdout:(35/156): ceph-test-19.0.0-3570.g10380055.el9.x 11 MB/s | 49 MB 00:04 2024-05-11T00:58:53.312 INFO:teuthology.orchestra.run.smithi119.stdout:(36/156): ceph-volume-19.0.0-3570.g10380055.el9 295 kB/s | 273 kB 00:00 2024-05-11T00:58:53.403 INFO:teuthology.orchestra.run.smithi119.stdout:(37/156): cephfs-top-19.0.0-3570.g10380055.el9. 199 kB/s | 18 kB 00:00 2024-05-11T00:58:53.434 INFO:teuthology.orchestra.run.smithi052.stdout:(7/156): ceph-mon-19.0.0-3570.g10380055.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-05-11T00:58:53.496 INFO:teuthology.orchestra.run.smithi119.stdout:(38/156): cephadm-19.0.0-3570.g10380055.el9.noa 3.6 MB/s | 763 kB 00:00 2024-05-11T00:58:53.579 INFO:teuthology.orchestra.run.smithi119.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 458 kB/s | 80 kB 00:00 2024-05-11T00:58:53.663 INFO:teuthology.orchestra.run.smithi119.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-11T00:58:53.763 INFO:teuthology.orchestra.run.smithi119.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-05-11T00:58:53.888 INFO:teuthology.orchestra.run.smithi119.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-05-11T00:58:53.947 INFO:teuthology.orchestra.run.smithi119.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-11T00:58:54.005 INFO:teuthology.orchestra.run.smithi119.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-11T00:58:54.039 INFO:teuthology.orchestra.run.smithi119.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-05-11T00:58:54.066 INFO:teuthology.orchestra.run.smithi119.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-11T00:58:54.090 INFO:teuthology.orchestra.run.smithi119.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-11T00:58:54.265 INFO:teuthology.orchestra.run.smithi119.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-05-11T00:58:54.307 INFO:teuthology.orchestra.run.smithi119.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-11T00:58:54.349 INFO:teuthology.orchestra.run.smithi119.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-05-11T00:58:54.399 INFO:teuthology.orchestra.run.smithi119.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-11T00:58:54.424 INFO:teuthology.orchestra.run.smithi119.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-05-11T00:58:54.458 INFO:teuthology.orchestra.run.smithi119.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 554 kB/s | 33 kB 00:00 2024-05-11T00:58:54.500 INFO:teuthology.orchestra.run.smithi119.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-05-11T00:58:54.542 INFO:teuthology.orchestra.run.smithi119.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-05-11T00:58:54.584 INFO:teuthology.orchestra.run.smithi119.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-05-11T00:58:54.626 INFO:teuthology.orchestra.run.smithi119.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-11T00:58:54.685 INFO:teuthology.orchestra.run.smithi119.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-11T00:58:54.743 INFO:teuthology.orchestra.run.smithi119.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-11T00:58:54.786 INFO:teuthology.orchestra.run.smithi119.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 612 kB/s | 25 kB 00:00 2024-05-11T00:58:54.827 INFO:teuthology.orchestra.run.smithi119.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-11T00:58:54.869 INFO:teuthology.orchestra.run.smithi119.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-05-11T00:58:54.920 INFO:teuthology.orchestra.run.smithi119.stdout:(63/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-05-11T00:58:54.945 INFO:teuthology.orchestra.run.smithi119.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-11T00:58:54.962 INFO:teuthology.orchestra.run.smithi119.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 841 kB/s | 35 kB 00:00 2024-05-11T00:58:55.246 INFO:teuthology.orchestra.run.smithi119.stdout:(66/156): ceph-mgr-dashboard-19.0.0-3570.g10380 898 kB/s | 3.5 MB 00:04 2024-05-11T00:58:55.304 INFO:teuthology.orchestra.run.smithi119.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-05-11T00:58:55.438 INFO:teuthology.orchestra.run.smithi119.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-05-11T00:58:55.488 INFO:teuthology.orchestra.run.smithi119.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-05-11T00:58:55.688 INFO:teuthology.orchestra.run.smithi119.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-05-11T00:58:55.839 INFO:teuthology.orchestra.run.smithi119.stdout:(71/156): postgresql-13.14-1.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-05-11T00:58:55.905 INFO:teuthology.orchestra.run.smithi119.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-11T00:58:55.964 INFO:teuthology.orchestra.run.smithi119.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.7 MB/s | 1.0 MB 00:00 2024-05-11T00:58:56.031 INFO:teuthology.orchestra.run.smithi119.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-05-11T00:58:56.131 INFO:teuthology.orchestra.run.smithi119.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 9.0 MB/s | 5.8 MB 00:00 2024-05-11T00:58:56.165 INFO:teuthology.orchestra.run.smithi119.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-05-11T00:58:56.191 INFO:teuthology.orchestra.run.smithi119.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-05-11T00:58:56.232 INFO:teuthology.orchestra.run.smithi119.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-05-11T00:58:56.257 INFO:teuthology.orchestra.run.smithi119.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-05-11T00:58:56.283 INFO:teuthology.orchestra.run.smithi119.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-05-11T00:58:56.342 INFO:teuthology.orchestra.run.smithi052.stdout:(8/156): ceph-radosgw-19.0.0-3570.g10380055.el9 3.6 MB/s | 11 MB 00:02 2024-05-11T00:58:56.358 INFO:teuthology.orchestra.run.smithi119.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-11T00:58:56.443 INFO:teuthology.orchestra.run.smithi052.stdout:(9/156): ceph-selinux-19.0.0-3570.g10380055.el9 252 kB/s | 25 kB 00:00 2024-05-11T00:58:56.476 INFO:teuthology.orchestra.run.smithi119.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-05-11T00:58:56.501 INFO:teuthology.orchestra.run.smithi119.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-05-11T00:58:56.526 INFO:teuthology.orchestra.run.smithi119.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-11T00:58:56.551 INFO:teuthology.orchestra.run.smithi119.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 838 kB/s | 42 kB 00:00 2024-05-11T00:58:56.602 INFO:teuthology.orchestra.run.smithi119.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-11T00:58:56.627 INFO:teuthology.orchestra.run.smithi119.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-11T00:58:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-11T00:58:56.677 INFO:teuthology.orchestra.run.smithi119.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-11T00:58:56.703 INFO:teuthology.orchestra.run.smithi119.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-11T00:58:56.770 INFO:teuthology.orchestra.run.smithi119.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-05-11T00:58:56.795 INFO:teuthology.orchestra.run.smithi119.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 452 kB/s | 42 kB 00:00 2024-05-11T00:58:56.820 INFO:teuthology.orchestra.run.smithi119.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-05-11T00:58:56.846 INFO:teuthology.orchestra.run.smithi119.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-11T00:58:56.871 INFO:teuthology.orchestra.run.smithi119.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-05-11T00:58:56.896 INFO:teuthology.orchestra.run.smithi119.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-05-11T00:58:56.921 INFO:teuthology.orchestra.run.smithi119.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-11T00:58:56.939 INFO:teuthology.orchestra.run.smithi119.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-05-11T00:58:56.964 INFO:teuthology.orchestra.run.smithi119.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-05-11T00:58:56.989 INFO:teuthology.orchestra.run.smithi119.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-11T00:58:56.993 INFO:teuthology.orchestra.run.smithi052.stdout:(10/156): ceph-osd-19.0.0-3570.g10380055.el9.x8 4.1 MB/s | 17 MB 00:04 2024-05-11T00:58:57.014 INFO:teuthology.orchestra.run.smithi119.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-05-11T00:58:57.085 INFO:teuthology.orchestra.run.smithi052.stdout:(11/156): cephfs-mirror-19.0.0-3570.g10380055.e 2.4 MB/s | 224 kB 00:00 2024-05-11T00:58:57.123 INFO:teuthology.orchestra.run.smithi119.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-05-11T00:58:57.177 INFO:teuthology.orchestra.run.smithi052.stdout:(12/156): cephfs-shell-19.0.0-3570.g10380055.el 295 kB/s | 27 kB 00:00 2024-05-11T00:58:57.294 INFO:teuthology.orchestra.run.smithi052.stdout:(13/156): libcephfs-devel-19.0.0-3570.g10380055 271 kB/s | 32 kB 00:00 2024-05-11T00:58:57.307 INFO:teuthology.orchestra.run.smithi119.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-05-11T00:58:57.711 INFO:teuthology.orchestra.run.smithi052.stdout:(14/156): libcephfs2-19.0.0-3570.g10380055.el9. 1.8 MB/s | 747 kB 00:00 2024-05-11T00:58:57.724 INFO:teuthology.orchestra.run.smithi119.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 913 kB/s | 548 kB 00:00 2024-05-11T00:58:57.811 INFO:teuthology.orchestra.run.smithi052.stdout:(15/156): libcephsqlite-19.0.0-3570.g10380055.e 1.6 MB/s | 164 kB 00:00 2024-05-11T00:58:57.815 INFO:teuthology.orchestra.run.smithi119.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-05-11T00:58:57.841 INFO:teuthology.orchestra.run.smithi119.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 579 kB/s | 309 kB 00:00 2024-05-11T00:58:57.911 INFO:teuthology.orchestra.run.smithi052.stdout:(16/156): librados-devel-19.0.0-3570.g10380055. 1.2 MB/s | 127 kB 00:00 2024-05-11T00:58:57.941 INFO:teuthology.orchestra.run.smithi119.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-05-11T00:58:58.050 INFO:teuthology.orchestra.run.smithi119.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-05-11T00:58:58.145 INFO:teuthology.orchestra.run.smithi052.stdout:(17/156): ceph-common-19.0.0-3570.g10380055.el9 2.7 MB/s | 21 MB 00:07 2024-05-11T00:58:58.150 INFO:teuthology.orchestra.run.smithi119.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 673 kB/s | 67 kB 00:00 2024-05-11T00:58:58.217 INFO:teuthology.orchestra.run.smithi119.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-11T00:58:58.242 INFO:teuthology.orchestra.run.smithi052.stdout:(18/156): libradosstriper1-19.0.0-3570.g1038005 1.5 MB/s | 501 kB 00:00 2024-05-11T00:58:58.276 INFO:teuthology.orchestra.run.smithi119.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-05-11T00:58:58.335 INFO:teuthology.orchestra.run.smithi119.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-11T00:58:58.337 INFO:teuthology.orchestra.run.smithi052.stdout:(19/156): python3-ceph-argparse-19.0.0-3570.g10 477 kB/s | 45 kB 00:00 2024-05-11T00:58:58.437 INFO:teuthology.orchestra.run.smithi052.stdout:(20/156): python3-ceph-common-19.0.0-3570.g1038 1.3 MB/s | 135 kB 00:00 2024-05-11T00:58:58.538 INFO:teuthology.orchestra.run.smithi052.stdout:(21/156): python3-cephfs-19.0.0-3570.g10380055. 1.6 MB/s | 161 kB 00:00 2024-05-11T00:58:58.543 INFO:teuthology.orchestra.run.smithi119.stdout:(113/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.7 MB/s | 16 MB 00:01 2024-05-11T00:58:58.568 INFO:teuthology.orchestra.run.smithi119.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 186 kB/s | 43 kB 00:00 2024-05-11T00:58:58.643 INFO:teuthology.orchestra.run.smithi119.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-05-11T00:58:58.686 INFO:teuthology.orchestra.run.smithi119.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-05-11T00:58:58.713 INFO:teuthology.orchestra.run.smithi052.stdout:(22/156): python3-rados-19.0.0-3570.g10380055.e 1.8 MB/s | 321 kB 00:00 2024-05-11T00:58:58.786 INFO:teuthology.orchestra.run.smithi119.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-05-11T00:58:58.880 INFO:teuthology.orchestra.run.smithi052.stdout:(23/156): python3-rbd-19.0.0-3570.g10380055.el9 1.8 MB/s | 300 kB 00:00 2024-05-11T00:58:58.886 INFO:teuthology.orchestra.run.smithi119.stdout:(118/156): python3-cachetools-4.2.4-1.el9.noarc 94 kB/s | 32 kB 00:00 2024-05-11T00:58:58.931 INFO:teuthology.orchestra.run.smithi052.stdout:(24/156): python3-rgw-19.0.0-3570.g10380055.el9 1.9 MB/s | 99 kB 00:00 2024-05-11T00:58:58.936 INFO:teuthology.orchestra.run.smithi119.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-05-11T00:58:58.989 INFO:teuthology.orchestra.run.smithi052.stdout:(25/156): rbd-fuse-19.0.0-3570.g10380055.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-11T00:58:59.053 INFO:teuthology.orchestra.run.smithi119.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-11T00:58:59.079 INFO:teuthology.orchestra.run.smithi119.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 183 kB/s | 35 kB 00:00 2024-05-11T00:58:59.112 INFO:teuthology.orchestra.run.smithi119.stdout:(122/156): python3-cherrypy-18.6.1-2.el9.noarch 841 kB/s | 358 kB 00:00 2024-05-11T00:58:59.204 INFO:teuthology.orchestra.run.smithi119.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-11T00:58:59.304 INFO:teuthology.orchestra.run.smithi119.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-05-11T00:58:59.355 INFO:teuthology.orchestra.run.smithi119.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-05-11T00:58:59.382 INFO:teuthology.orchestra.run.smithi119.stdout:(126/156): python3-grpcio-tools-1.46.7-10.el9.x 480 kB/s | 144 kB 00:00 2024-05-11T00:58:59.405 INFO:teuthology.orchestra.run.smithi119.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-11T00:58:59.447 INFO:teuthology.orchestra.run.smithi119.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 210 kB/s | 19 kB 00:00 2024-05-11T00:58:59.473 INFO:teuthology.orchestra.run.smithi052.stdout:(26/156): librgw2-19.0.0-3570.g10380055.el9.x86 4.0 MB/s | 5.4 MB 00:01 2024-05-11T00:58:59.556 INFO:teuthology.orchestra.run.smithi052.stdout:(27/156): rbd-mirror-19.0.0-3570.g10380055.el9. 5.5 MB/s | 3.1 MB 00:00 2024-05-11T00:58:59.573 INFO:teuthology.orchestra.run.smithi119.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 102 kB/s | 20 kB 00:00 2024-05-11T00:58:59.573 INFO:teuthology.orchestra.run.smithi052.stdout:(28/156): rbd-nbd-19.0.0-3570.g10380055.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-11T00:58:59.598 INFO:teuthology.orchestra.run.smithi119.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 307 kB/s | 46 kB 00:00 2024-05-11T00:58:59.607 INFO:teuthology.orchestra.run.smithi052.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3570.g 682 kB/s | 34 kB 00:00 2024-05-11T00:58:59.632 INFO:teuthology.orchestra.run.smithi052.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3570.g1038005 2.4 MB/s | 142 kB 00:00 2024-05-11T00:58:59.640 INFO:teuthology.orchestra.run.smithi119.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-05-11T00:58:59.699 INFO:teuthology.orchestra.run.smithi119.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-05-11T00:58:59.724 INFO:teuthology.orchestra.run.smithi119.stdout:(133/156): python3-more-itertools-8.12.0-2.el9. 522 kB/s | 79 kB 00:00 2024-05-11T00:58:59.749 INFO:teuthology.orchestra.run.smithi119.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-11T00:58:59.792 INFO:teuthology.orchestra.run.smithi119.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 177 kB/s | 16 kB 00:00 2024-05-11T00:58:59.842 INFO:teuthology.orchestra.run.smithi119.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 766 kB/s | 90 kB 00:00 2024-05-11T00:58:59.867 INFO:teuthology.orchestra.run.smithi119.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 240 kB/s | 28 kB 00:00 2024-05-11T00:58:59.892 INFO:teuthology.orchestra.run.smithi119.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-05-11T00:59:00.035 INFO:teuthology.orchestra.run.smithi119.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 354 kB/s | 59 kB 00:00 2024-05-11T00:59:00.060 INFO:teuthology.orchestra.run.smithi119.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 214 kB/s | 36 kB 00:00 2024-05-11T00:59:00.077 INFO:teuthology.orchestra.run.smithi119.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 804 kB/s | 188 kB 00:00 2024-05-11T00:59:00.169 INFO:teuthology.orchestra.run.smithi119.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 381 kB/s | 51 kB 00:00 2024-05-11T00:59:00.194 INFO:teuthology.orchestra.run.smithi119.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-05-11T00:59:00.219 INFO:teuthology.orchestra.run.smithi119.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 630 kB/s | 90 kB 00:00 2024-05-11T00:59:00.269 INFO:teuthology.orchestra.run.smithi119.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-11T00:59:00.286 INFO:teuthology.orchestra.run.smithi119.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-05-11T00:59:00.378 INFO:teuthology.orchestra.run.smithi119.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-05-11T00:59:00.516 INFO:teuthology.orchestra.run.smithi052.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3570.g10380 3.9 MB/s | 3.5 MB 00:00 2024-05-11T00:59:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-05-11T00:59:00.641 INFO:teuthology.orchestra.run.smithi052.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3570.g10 2.1 MB/s | 274 kB 00:00 2024-05-11T00:59:00.725 INFO:teuthology.orchestra.run.smithi052.stdout:(33/156): ceph-mgr-rook-19.0.0-3570.g10380055.e 593 kB/s | 49 kB 00:00 2024-05-11T00:59:00.771 INFO:teuthology.orchestra.run.smithi119.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 9.2 MB/s | 2.3 MB 00:00 2024-05-11T00:59:00.854 INFO:teuthology.orchestra.run.smithi119.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-05-11T00:59:00.904 INFO:teuthology.orchestra.run.smithi119.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 837 kB/s | 42 kB 00:00 2024-05-11T00:59:00.971 INFO:teuthology.orchestra.run.smithi119.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 3.7 MB/s | 248 kB 00:00 2024-05-11T00:59:01.022 INFO:teuthology.orchestra.run.smithi119.stdout:(153/156): librados2-19.0.0-3570.g10380055.el9. 4.6 MB/s | 3.4 MB 00:00 2024-05-11T00:59:01.042 INFO:teuthology.orchestra.run.smithi052.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3570.g1 52 kB/s | 16 kB 00:00 2024-05-11T00:59:01.055 INFO:teuthology.orchestra.run.smithi119.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-05-11T00:59:01.150 INFO:teuthology.orchestra.run.smithi052.stdout:(35/156): ceph-volume-19.0.0-3570.g10380055.el9 2.5 MB/s | 273 kB 00:00 2024-05-11T00:59:01.280 INFO:teuthology.orchestra.run.smithi119.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-05-11T00:59:01.434 INFO:teuthology.orchestra.run.smithi052.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-05-11T00:59:01.460 INFO:teuthology.orchestra.run.smithi052.stdout:(37/156): cephadm-19.0.0-3570.g10380055.el9.noa 2.4 MB/s | 763 kB 00:00 2024-05-11T00:59:01.493 INFO:teuthology.orchestra.run.smithi052.stdout:(38/156): cephfs-top-19.0.0-3570.g10380055.el9. 312 kB/s | 18 kB 00:00 2024-05-11T00:59:01.568 INFO:teuthology.orchestra.run.smithi052.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 741 kB/s | 80 kB 00:00 2024-05-11T00:59:01.636 INFO:teuthology.orchestra.run.smithi052.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-11T00:59:01.685 INFO:teuthology.orchestra.run.smithi052.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-05-11T00:59:01.719 INFO:teuthology.orchestra.run.smithi052.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-05-11T00:59:01.778 INFO:teuthology.orchestra.run.smithi052.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-05-11T00:59:01.854 INFO:teuthology.orchestra.run.smithi052.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 794 kB/s | 106 kB 00:00 2024-05-11T00:59:01.904 INFO:teuthology.orchestra.run.smithi052.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-11T00:59:01.929 INFO:teuthology.orchestra.run.smithi052.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-05-11T00:59:01.954 INFO:teuthology.orchestra.run.smithi052.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-11T00:59:02.104 INFO:teuthology.orchestra.run.smithi052.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-05-11T00:59:02.146 INFO:teuthology.orchestra.run.smithi052.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-05-11T00:59:02.172 INFO:teuthology.orchestra.run.smithi052.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-11T00:59:02.197 INFO:teuthology.orchestra.run.smithi052.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-05-11T00:59:02.222 INFO:teuthology.orchestra.run.smithi052.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-11T00:59:02.247 INFO:teuthology.orchestra.run.smithi052.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-05-11T00:59:02.290 INFO:teuthology.orchestra.run.smithi052.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-05-11T00:59:02.348 INFO:teuthology.orchestra.run.smithi052.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-05-11T00:59:02.373 INFO:teuthology.orchestra.run.smithi052.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-11T00:59:02.399 INFO:teuthology.orchestra.run.smithi052.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-11T00:59:02.424 INFO:teuthology.orchestra.run.smithi052.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-11T00:59:02.466 INFO:teuthology.orchestra.run.smithi052.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-11T00:59:02.500 INFO:teuthology.orchestra.run.smithi052.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-05-11T00:59:02.525 INFO:teuthology.orchestra.run.smithi052.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-05-11T00:59:02.550 INFO:teuthology.orchestra.run.smithi052.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-05-11T00:59:02.576 INFO:teuthology.orchestra.run.smithi052.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-11T00:59:02.601 INFO:teuthology.orchestra.run.smithi052.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-11T00:59:02.626 INFO:teuthology.orchestra.run.smithi052.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-05-11T00:59:02.743 INFO:teuthology.orchestra.run.smithi052.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-05-11T00:59:02.802 INFO:teuthology.orchestra.run.smithi052.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-05-11T00:59:02.927 INFO:teuthology.orchestra.run.smithi052.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-05-11T00:59:02.969 INFO:teuthology.orchestra.run.smithi052.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-05-11T00:59:02.994 INFO:teuthology.orchestra.run.smithi052.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-11T00:59:03.062 INFO:teuthology.orchestra.run.smithi052.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-11T00:59:03.145 INFO:teuthology.orchestra.run.smithi052.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-05-11T00:59:03.164 INFO:teuthology.orchestra.run.smithi119.stdout:(156/156): librbd1-19.0.0-3570.g10380055.el9.x8 1.1 MB/s | 3.1 MB 00:02 2024-05-11T00:59:03.166 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-11T00:59:03.167 INFO:teuthology.orchestra.run.smithi119.stdout:Total 13 MB/s | 226 MB 00:17 2024-05-11T00:59:03.170 INFO:teuthology.orchestra.run.smithi052.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 805 kB/s | 87 kB 00:00 2024-05-11T00:59:03.212 INFO:teuthology.orchestra.run.smithi052.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-11T00:59:03.296 INFO:teuthology.orchestra.run.smithi052.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-05-11T00:59:03.321 INFO:teuthology.orchestra.run.smithi052.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-11T00:59:03.338 INFO:teuthology.orchestra.run.smithi052.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-05-11T00:59:03.355 INFO:teuthology.orchestra.run.smithi052.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 576 kB/s | 19 kB 00:00 2024-05-11T00:59:03.380 INFO:teuthology.orchestra.run.smithi052.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-11T00:59:03.406 INFO:teuthology.orchestra.run.smithi052.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-05-11T00:59:03.448 INFO:teuthology.orchestra.run.smithi052.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-05-11T00:59:03.523 INFO:teuthology.orchestra.run.smithi052.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-05-11T00:59:03.548 INFO:teuthology.orchestra.run.smithi052.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-11T00:59:03.573 INFO:teuthology.orchestra.run.smithi052.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-11T00:59:03.599 INFO:teuthology.orchestra.run.smithi052.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-05-11T00:59:03.624 INFO:teuthology.orchestra.run.smithi052.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-11T00:59:03.649 INFO:teuthology.orchestra.run.smithi052.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-11T00:59:03.675 INFO:teuthology.orchestra.run.smithi052.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-11T00:59:03.700 INFO:teuthology.orchestra.run.smithi052.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-11T00:59:03.725 INFO:teuthology.orchestra.run.smithi052.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-11T00:59:03.767 INFO:teuthology.orchestra.run.smithi052.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-11T00:59:03.809 INFO:teuthology.orchestra.run.smithi052.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-11T00:59:03.851 INFO:teuthology.orchestra.run.smithi052.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-05-11T00:59:03.885 INFO:teuthology.orchestra.run.smithi052.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-05-11T00:59:03.918 INFO:teuthology.orchestra.run.smithi052.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-11T00:59:04.077 INFO:teuthology.orchestra.run.smithi052.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-05-11T00:59:04.219 INFO:teuthology.orchestra.run.smithi052.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 305 kB 00:00 2024-05-11T00:59:04.244 INFO:teuthology.orchestra.run.smithi052.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 347 kB/s | 58 kB 00:00 2024-05-11T00:59:04.270 INFO:teuthology.orchestra.run.smithi052.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-11T00:59:04.296 INFO:teuthology.orchestra.run.smithi052.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-11T00:59:04.321 INFO:teuthology.orchestra.run.smithi052.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-05-11T00:59:04.379 INFO:teuthology.orchestra.run.smithi052.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-05-11T00:59:04.384 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-11T00:59:04.488 INFO:teuthology.orchestra.run.smithi052.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-05-11T00:59:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-05-11T00:59:04.532 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-11T00:59:04.533 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-11T00:59:04.547 INFO:teuthology.orchestra.run.smithi052.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 587 kB/s | 19 kB 00:00 2024-05-11T00:59:04.605 INFO:teuthology.orchestra.run.smithi052.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-05-11T00:59:04.639 INFO:teuthology.orchestra.run.smithi052.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 784 kB/s | 26 kB 00:00 2024-05-11T00:59:04.673 INFO:teuthology.orchestra.run.smithi052.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-05-11T00:59:04.740 INFO:teuthology.orchestra.run.smithi052.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-05-11T00:59:04.765 INFO:teuthology.orchestra.run.smithi052.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 734 kB/s | 67 kB 00:00 2024-05-11T00:59:04.790 INFO:teuthology.orchestra.run.smithi052.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-05-11T00:59:04.840 INFO:teuthology.orchestra.run.smithi052.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-05-11T00:59:04.866 INFO:teuthology.orchestra.run.smithi052.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-05-11T00:59:04.891 INFO:teuthology.orchestra.run.smithi052.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-05-11T00:59:04.917 INFO:teuthology.orchestra.run.smithi052.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-05-11T00:59:04.942 INFO:teuthology.orchestra.run.smithi052.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-05-11T00:59:04.967 INFO:teuthology.orchestra.run.smithi052.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-05-11T00:59:04.992 INFO:teuthology.orchestra.run.smithi052.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-05-11T00:59:05.017 INFO:teuthology.orchestra.run.smithi052.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-05-11T00:59:05.043 INFO:teuthology.orchestra.run.smithi052.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-05-11T00:59:05.068 INFO:teuthology.orchestra.run.smithi052.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-11T00:59:05.102 INFO:teuthology.orchestra.run.smithi052.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-05-11T00:59:05.152 INFO:teuthology.orchestra.run.smithi052.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-05-11T00:59:05.177 INFO:teuthology.orchestra.run.smithi052.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-11T00:59:05.203 INFO:teuthology.orchestra.run.smithi052.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-05-11T00:59:05.228 INFO:teuthology.orchestra.run.smithi052.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-05-11T00:59:05.253 INFO:teuthology.orchestra.run.smithi052.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-05-11T00:59:05.279 INFO:teuthology.orchestra.run.smithi052.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 389 kB/s | 20 kB 00:00 2024-05-11T00:59:05.321 INFO:teuthology.orchestra.run.smithi052.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-05-11T00:59:05.338 INFO:teuthology.orchestra.run.smithi052.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 792 kB/s | 46 kB 00:00 2024-05-11T00:59:05.363 INFO:teuthology.orchestra.run.smithi052.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-05-11T00:59:05.380 INFO:teuthology.orchestra.run.smithi052.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-05-11T00:59:05.414 INFO:teuthology.orchestra.run.smithi052.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-11T00:59:05.430 INFO:teuthology.orchestra.run.smithi052.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-11T00:59:05.455 INFO:teuthology.orchestra.run.smithi052.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-05-11T00:59:05.481 INFO:teuthology.orchestra.run.smithi052.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-05-11T00:59:05.506 INFO:teuthology.orchestra.run.smithi052.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-05-11T00:59:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-05-11T00:59:05.557 INFO:teuthology.orchestra.run.smithi052.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-11T00:59:05.582 INFO:teuthology.orchestra.run.smithi052.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-05-11T00:59:05.608 INFO:teuthology.orchestra.run.smithi052.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-11T00:59:05.633 INFO:teuthology.orchestra.run.smithi052.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-05-11T00:59:05.658 INFO:teuthology.orchestra.run.smithi052.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-11T00:59:06.100 INFO:teuthology.orchestra.run.smithi052.stdout:(144/156): ceph-test-19.0.0-3570.g10380055.el9. 5.1 MB/s | 49 MB 00:09 2024-05-11T00:59:06.125 INFO:teuthology.orchestra.run.smithi052.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 43 kB/s | 20 kB 00:00 2024-05-11T00:59:06.151 INFO:teuthology.orchestra.run.smithi052.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 826 kB/s | 427 kB 00:00 2024-05-11T00:59:06.209 INFO:teuthology.orchestra.run.smithi052.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-05-11T00:59:06.260 INFO:teuthology.orchestra.run.smithi052.stdout:(148/156): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-05-11T00:59:06.460 INFO:teuthology.orchestra.run.smithi052.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-05-11T00:59:06.552 INFO:teuthology.orchestra.run.smithi052.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-05-11T00:59:06.593 INFO:teuthology.orchestra.run.smithi052.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-05-11T00:59:06.644 INFO:teuthology.orchestra.run.smithi052.stdout:(152/156): librados2-19.0.0-3570.g10380055.el9. 6.9 MB/s | 3.4 MB 00:00 2024-05-11T00:59:06.669 INFO:teuthology.orchestra.run.smithi052.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 3.2 MB/s | 248 kB 00:00 2024-05-11T00:59:06.673 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-11T00:59:06.673 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-11T00:59:06.762 INFO:teuthology.orchestra.run.smithi052.stdout:(154/156): librbd1-19.0.0-3570.g10380055.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-05-11T00:59:06.836 INFO:teuthology.orchestra.run.smithi052.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 1.3 MB 00:00 2024-05-11T00:59:06.903 INFO:teuthology.orchestra.run.smithi052.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-05-11T00:59:06.907 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-05-11T00:59:06.908 INFO:teuthology.orchestra.run.smithi052.stdout:Total 13 MB/s | 226 MB 00:17 2024-05-11T00:59:08.104 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-05-11T00:59:08.272 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-05-11T00:59:08.272 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-05-11T00:59:10.545 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-05-11T00:59:10.545 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-05-11T00:59:12.871 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-11T00:59:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T00:59:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T00:59:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T00:59:13.111 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T00:59:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T00:59:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T00:59:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T00:59:13.654 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.0.0-3570.g10380055.el9.x86_64 8/164 2024-05-11T00:59:13.785 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.0.0-3570.g10380055.el9.x86_64 8/164 2024-05-11T00:59:13.813 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.0.0-3570.g10380055.el9.x86_64 9/164 2024-05-11T00:59:13.905 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.0.0-3570.g10380055.el9.x86_64 9/164 2024-05-11T00:59:13.935 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.0.0-3570.g10380055.el9.x86_64 10/164 2024-05-11T00:59:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T00:59:14.169 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T00:59:14.193 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.0.0-3570.g10380055.el9.x86_64 13/164 2024-05-11T00:59:14.279 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3570.g10380055.el9.x86_64 13/164 2024-05-11T00:59:14.302 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.0.0-3570.g10380055.el9.x86 14/164 2024-05-11T00:59:14.348 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3570.g10380055.el9.x86 14/164 2024-05-11T00:59:14.370 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T00:59:14.393 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T00:59:14.588 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T00:59:14.768 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T00:59:14.892 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T00:59:14.936 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T00:59:15.059 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T00:59:15.146 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T00:59:15.339 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T00:59:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T00:59:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T00:59:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T00:59:15.574 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T00:59:15.609 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T00:59:15.652 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T00:59:15.686 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T00:59:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T00:59:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T00:59:15.835 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T00:59:15.867 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T00:59:15.910 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T00:59:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T00:59:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.0.0-3570.g10380055.el9. 32/164 2024-05-11T00:59:16.064 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.0.0-3570.g10380055.el 33/164 2024-05-11T00:59:16.093 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.0.0-3570.g10380055.el9.x86_6 34/164 2024-05-11T00:59:16.163 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T00:59:16.216 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T00:59:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T00:59:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T00:59:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T00:59:16.885 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-05-11T00:59:16.979 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T00:59:17.014 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T00:59:17.017 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T00:59:17.082 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T00:59:17.107 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T00:59:17.118 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T00:59:17.135 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T00:59:17.143 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T00:59:17.180 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T00:59:17.197 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T00:59:17.217 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.0.0-3570.g10380055.el9.x86_6 44/164 2024-05-11T00:59:17.259 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T00:59:17.413 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T00:59:17.470 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T00:59:17.544 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T00:59:17.560 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T00:59:17.625 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T00:59:17.698 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T00:59:17.729 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T00:59:17.765 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T00:59:17.778 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librados2-2:19.0.0-3570.g10380055.el9.x86_64 8/164 2024-05-11T00:59:17.812 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T00:59:17.856 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T00:59:17.900 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T00:59:17.919 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:19.0.0-3570.g10380055.el9.x86_64 8/164 2024-05-11T00:59:17.942 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T00:59:17.945 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs2-2:19.0.0-3570.g10380055.el9.x86_64 9/164 2024-05-11T00:59:18.038 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephfs2-2:19.0.0-3570.g10380055.el9.x86_64 9/164 2024-05-11T00:59:18.066 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rados-2:19.0.0-3570.g10380055.el9.x86_64 10/164 2024-05-11T00:59:18.078 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T00:59:18.122 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T00:59:18.152 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T00:59:18.194 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T00:59:18.252 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T00:59:18.290 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T00:59:18.319 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T00:59:18.319 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T00:59:18.341 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephsqlite-2:19.0.0-3570.g10380055.el9.x86_64 13/164 2024-05-11T00:59:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T00:59:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T00:59:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T00:59:18.446 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3570.g10380055.el9.x86_64 13/164 2024-05-11T00:59:18.475 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libradosstriper1-2:19.0.0-3570.g10380055.el9.x86 14/164 2024-05-11T00:59:18.538 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3570.g10380055.el9.x86 14/164 2024-05-11T00:59:18.559 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T00:59:18.585 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T00:59:18.729 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T00:59:18.734 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T00:59:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T00:59:18.896 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T00:59:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T00:59:18.954 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T00:59:18.988 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T00:59:19.027 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T00:59:19.049 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T00:59:19.059 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T00:59:19.089 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T00:59:19.109 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T00:59:19.123 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T00:59:19.156 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T00:59:19.198 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T00:59:19.233 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T00:59:19.234 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T00:59:19.310 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T00:59:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T00:59:19.364 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T00:59:19.402 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T00:59:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T00:59:19.510 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T00:59:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T00:59:19.596 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T00:59:19.620 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T00:59:19.657 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T00:59:19.691 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T00:59:19.767 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T00:59:19.808 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T00:59:19.851 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T00:59:19.884 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T00:59:19.911 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T00:59:19.982 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T00:59:20.034 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T00:59:20.064 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T00:59:20.108 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T00:59:20.187 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T00:59:20.330 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T00:59:20.344 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-common-2:19.0.0-3570.g10380055.el9. 32/164 2024-05-11T00:59:20.376 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T00:59:20.388 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-argparse-2:19.0.0-3570.g10380055.el 33/164 2024-05-11T00:59:20.401 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T00:59:20.415 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cephfs-2:19.0.0-3570.g10380055.el9.x86_6 34/164 2024-05-11T00:59:20.489 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T00:59:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T00:59:20.551 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T00:59:20.556 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T00:59:20.634 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T00:59:20.637 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T00:59:20.687 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T00:59:20.779 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T00:59:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T00:59:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T00:59:20.909 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T00:59:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T00:59:21.463 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T00:59:21.548 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T00:59:21.583 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T00:59:21.626 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T00:59:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T00:59:21.664 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librados-devel-2:19.0.0-3570.g10380055.el9.x86_6 44/164 2024-05-11T00:59:21.706 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T00:59:21.741 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T00:59:21.861 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T00:59:21.933 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T00:59:22.028 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T00:59:22.091 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T00:59:22.164 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T00:59:22.201 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T00:59:22.238 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T00:59:22.380 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T00:59:22.513 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T00:59:22.557 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T00:59:22.599 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T00:59:22.742 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T00:59:22.787 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T00:59:22.825 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T00:59:22.867 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T00:59:22.964 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T00:59:22.996 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T00:59:23.000 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T00:59:23.042 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T00:59:23.074 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T00:59:23.098 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T00:59:23.106 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T00:59:23.133 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T00:59:23.166 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T00:59:23.196 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T00:59:23.330 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T00:59:23.364 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T00:59:23.393 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T00:59:23.427 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T00:59:23.466 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T00:59:23.481 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T00:59:23.506 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T00:59:23.521 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.0.0-3570.g10380055.el9.x86_64 101/164 2024-05-11T00:59:23.536 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T00:59:23.570 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T00:59:23.603 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T00:59:23.612 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.0.0-3570.g10380055.el9.x86_64 101/164 2024-05-11T00:59:23.644 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.0.0-3570.g10380055.el9.x86_64 102/164 2024-05-11T00:59:23.645 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T00:59:23.680 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T00:59:23.685 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T00:59:23.769 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T00:59:23.828 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T00:59:23.858 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T00:59:23.905 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T00:59:24.007 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T00:59:24.060 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T00:59:24.165 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T00:59:24.191 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.0.0-3570.g10380055.el9.x86_64 105/164 2024-05-11T00:59:24.262 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.0.0-3570.g10380055.el9.x86_64 105/164 2024-05-11T00:59:24.284 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.0.0-3570.g10380055.el9.x86_64 106/164 2024-05-11T00:59:24.819 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T00:59:24.865 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T00:59:24.898 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T00:59:25.001 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T00:59:25.058 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T00:59:25.147 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T00:59:25.184 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T00:59:25.360 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T00:59:25.390 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T00:59:25.471 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T00:59:26.176 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T00:59:26.292 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T00:59:26.585 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 107/164 2024-05-11T00:59:26.623 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 107/164 2024-05-11T00:59:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 107/164 2024-05-11T00:59:27.210 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.0.0-3570.g10380055.el9.x86_64 108/164 2024-05-11T00:59:27.309 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.0.0-3570.g10380055.el9.x86_64 108/164 2024-05-11T00:59:27.309 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T00:59:27.309 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T00:59:27.309 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:27.353 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.0.0-3570.g10380055.el9.x86_64 109/164 2024-05-11T00:59:27.630 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T00:59:27.702 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T00:59:27.731 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T00:59:27.763 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T00:59:28.092 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T00:59:28.143 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librbd1-2:19.0.0-3570.g10380055.el9.x86_64 101/164 2024-05-11T00:59:28.234 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:19.0.0-3570.g10380055.el9.x86_64 101/164 2024-05-11T00:59:28.256 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rbd-2:19.0.0-3570.g10380055.el9.x86_64 102/164 2024-05-11T00:59:28.293 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T00:59:28.802 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T00:59:28.829 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librgw2-2:19.0.0-3570.g10380055.el9.x86_64 105/164 2024-05-11T00:59:28.901 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librgw2-2:19.0.0-3570.g10380055.el9.x86_64 105/164 2024-05-11T00:59:28.931 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rgw-2:19.0.0-3570.g10380055.el9.x86_64 106/164 2024-05-11T00:59:31.358 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 107/164 2024-05-11T00:59:31.402 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 107/164 2024-05-11T00:59:31.976 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 107/164 2024-05-11T00:59:32.014 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-base-2:19.0.0-3570.g10380055.el9.x86_64 108/164 2024-05-11T00:59:32.106 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-base-2:19.0.0-3570.g10380055.el9.x86_64 108/164 2024-05-11T00:59:32.106 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T00:59:32.106 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T00:59:32.106 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-selinux-2:19.0.0-3570.g10380055.el9.x86_64 109/164 2024-05-11T00:59:39.139 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3570.g10380055.el9.x86_64 109/164 2024-05-11T00:59:39.139 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-05-11T00:59:39.139 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-05-11T00:59:39.139 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-05-11T00:59:39.139 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-05-11T00:59:39.139 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-05-11T00:59:39.140 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-05-11T00:59:39.140 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-05-11T00:59:39.140 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:39.312 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.0.0-3570.g10380055.el9.noarch 110/164 2024-05-11T00:59:39.328 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.0.0-3570.g10380055.el9.noarch 110/164 2024-05-11T00:59:39.328 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:39.328 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T00:59:39.328 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:40.815 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.0.0-3570.g10380055.el9.x86_64 111/164 2024-05-11T00:59:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.0.0-3570.g10380055.el9.x86_64 111/164 2024-05-11T00:59:40.843 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:40.843 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T00:59:40.843 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T00:59:40.843 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T00:59:40.843 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:41.066 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.0.0-3570.g10380055.el9.x86_64 112/164 2024-05-11T00:59:41.089 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.0.0-3570.g10380055.el9.x86_64 112/164 2024-05-11T00:59:41.090 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:41.090 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T00:59:41.090 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T00:59:41.090 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T00:59:41.090 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:41.525 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.0.0-3570.g10380055.el9.x86_64 113/164 2024-05-11T00:59:41.548 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.0.0-3570.g10380055.el9.x86_64 113/164 2024-05-11T00:59:41.549 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:41.549 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T00:59:41.549 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T00:59:41.549 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T00:59:41.549 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:41.655 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T00:59:41.712 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T00:59:41.767 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T00:59:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T00:59:42.009 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T00:59:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T00:59:42.176 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T00:59:42.316 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T00:59:42.368 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T00:59:42.400 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T00:59:42.427 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T00:59:42.830 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-3570.g10380055.el9.noarch 125/164 2024-05-11T00:59:42.859 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.0.0-3570.g10380055.el9.noarch 125/164 2024-05-11T00:59:42.891 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3570.g10380055.e 126/164 2024-05-11T00:59:42.953 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3570.g10380055. 127/164 2024-05-11T00:59:42.982 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3570.g10380055.el9.noa 128/164 2024-05-11T00:59:44.023 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3570.g10380055.el9.noa 128/164 2024-05-11T00:59:44.067 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3570.g10380055.el9.n 129/164 2024-05-11T00:59:44.131 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3570.g10380055.el9.x86_64 109/164 2024-05-11T00:59:44.131 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /sys 2024-05-11T00:59:44.132 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /proc 2024-05-11T00:59:44.132 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /mnt 2024-05-11T00:59:44.132 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /var/tmp 2024-05-11T00:59:44.132 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /home 2024-05-11T00:59:44.132 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /root 2024-05-11T00:59:44.132 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /tmp 2024-05-11T00:59:44.132 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:44.332 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-volume-2:19.0.0-3570.g10380055.el9.noarch 110/164 2024-05-11T00:59:44.349 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-volume-2:19.0.0-3570.g10380055.el9.noarch 110/164 2024-05-11T00:59:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T00:59:44.349 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:45.037 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3570.g10380055.el9.n 129/164 2024-05-11T00:59:45.059 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3570.g103 130/164 2024-05-11T00:59:45.172 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3570.g103 130/164 2024-05-11T00:59:45.339 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3570.g10380055.el 131/164 2024-05-11T00:59:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.0.0-3570.g10380055.el9.x86_64 132/164 2024-05-11T00:59:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3570.g10380055.el9.x86_64 132/164 2024-05-11T00:59:45.402 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:45.402 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T00:59:45.403 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T00:59:45.403 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T00:59:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.0.0-3570.g10380055.el9.noarch 133/164 2024-05-11T00:59:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3570.g10380055.el9.noarch 133/164 2024-05-11T00:59:45.563 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.0.0-3570.g10380055.el9.x86_64 134/164 2024-05-11T00:59:45.616 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.0.0-3570.g10380055.el9.x86_64 135/164 2024-05-11T00:59:45.894 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-osd-2:19.0.0-3570.g10380055.el9.x86_64 111/164 2024-05-11T00:59:45.922 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-osd-2:19.0.0-3570.g10380055.el9.x86_64 111/164 2024-05-11T00:59:45.922 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:45.922 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T00:59:45.922 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T00:59:45.922 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T00:59:45.922 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:46.161 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mds-2:19.0.0-3570.g10380055.el9.x86_64 112/164 2024-05-11T00:59:46.186 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mds-2:19.0.0-3570.g10380055.el9.x86_64 112/164 2024-05-11T00:59:46.186 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:46.186 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T00:59:46.186 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T00:59:46.186 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T00:59:46.186 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:46.542 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T00:59:46.573 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.0.0-3570.g10380055.el9.x86_64 137/164 2024-05-11T00:59:46.597 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3570.g10380055.el9.x86_64 137/164 2024-05-11T00:59:46.597 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:46.597 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T00:59:46.597 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T00:59:46.598 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T00:59:46.598 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:46.640 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3570.g10380 138/164 2024-05-11T00:59:46.653 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mon-2:19.0.0-3570.g10380055.el9.x86_64 113/164 2024-05-11T00:59:46.664 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3570.g10380 138/164 2024-05-11T00:59:46.664 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:46.664 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T00:59:46.664 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:46.678 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mon-2:19.0.0-3570.g10380055.el9.x86_64 113/164 2024-05-11T00:59:46.678 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:46.678 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T00:59:46.678 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T00:59:46.678 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T00:59:46.678 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:46.716 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:19.0.0-3570.g10380055.el9.x86_64 139/164 2024-05-11T00:59:46.739 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3570.g10380055.el9.x86_64 139/164 2024-05-11T00:59:46.739 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:46.739 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T00:59:46.739 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T00:59:46.740 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T00:59:46.740 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:46.784 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T00:59:46.842 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T00:59:46.906 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T00:59:47.041 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.0.0-3570.g10380055.el9.x86_64 140/164 2024-05-11T00:59:47.065 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3570.g10380055.el9.x86_64 140/164 2024-05-11T00:59:47.065 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:47.065 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T00:59:47.065 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T00:59:47.066 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T00:59:47.066 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T00:59:47.109 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T00:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T00:59:47.234 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T00:59:47.324 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T00:59:47.478 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T00:59:47.526 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T00:59:47.562 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T00:59:47.590 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T00:59:47.945 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephadm-2:19.0.0-3570.g10380055.el9.noarch 125/164 2024-05-11T00:59:47.973 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephadm-2:19.0.0-3570.g10380055.el9.noarch 125/164 2024-05-11T00:59:48.003 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3570.g10380055.e 126/164 2024-05-11T00:59:48.068 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3570.g10380055. 127/164 2024-05-11T00:59:48.094 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3570.g10380055.el9.noa 128/164 2024-05-11T00:59:49.188 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3570.g10380055.el9.noa 128/164 2024-05-11T00:59:49.221 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3570.g10380055.el9.n 129/164 2024-05-11T00:59:50.235 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3570.g10380055.el9.n 129/164 2024-05-11T00:59:50.263 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3570.g103 130/164 2024-05-11T00:59:50.381 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3570.g103 130/164 2024-05-11T00:59:50.548 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3570.g10380055.el 131/164 2024-05-11T00:59:50.582 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-2:19.0.0-3570.g10380055.el9.x86_64 132/164 2024-05-11T00:59:50.607 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3570.g10380055.el9.x86_64 132/164 2024-05-11T00:59:50.607 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:50.607 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T00:59:50.607 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T00:59:50.608 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T00:59:50.608 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:50.658 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-rook-2:19.0.0-3570.g10380055.el9.noarch 133/164 2024-05-11T00:59:50.669 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3570.g10380055.el9.noarch 133/164 2024-05-11T00:59:50.770 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-2:19.0.0-3570.g10380055.el9.x86_64 134/164 2024-05-11T00:59:50.824 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-fuse-2:19.0.0-3570.g10380055.el9.x86_64 135/164 2024-05-11T00:59:51.712 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.0.0-3570.g10380055.el9.x86_64 141/164 2024-05-11T00:59:51.753 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.0.0-3570.g10380055.el9.x86_64 142/164 2024-05-11T00:59:51.794 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.0.0-3570.g10380055.el9.x86_64 143/164 2024-05-11T00:59:51.797 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T00:59:51.826 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T00:59:51.827 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-radosgw-2:19.0.0-3570.g10380055.el9.x86_64 137/164 2024-05-11T00:59:51.852 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3570.g10380055.el9.x86_64 137/164 2024-05-11T00:59:51.853 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:51.853 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T00:59:51.853 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T00:59:51.853 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T00:59:51.853 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:51.853 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T00:59:51.894 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3570.g10380 138/164 2024-05-11T00:59:51.918 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3570.g10380 138/164 2024-05-11T00:59:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:51.918 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T00:59:51.918 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:51.970 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-mirror-2:19.0.0-3570.g10380055.el9.x86_64 139/164 2024-05-11T00:59:51.994 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3570.g10380055.el9.x86_64 139/164 2024-05-11T00:59:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T00:59:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T00:59:51.994 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T00:59:51.994 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:52.046 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-shell-2:19.0.0-3570.g10380055.el9.x86_64 146/164 2024-05-11T00:59:52.099 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T00:59:52.130 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.0.0-3570.g10380055.el9.x86_ 148/164 2024-05-11T00:59:52.160 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T00:59:52.197 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T00:59:52.312 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-mirror-2:19.0.0-3570.g10380055.el9.x86_64 140/164 2024-05-11T00:59:52.336 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3570.g10380055.el9.x86_64 140/164 2024-05-11T00:59:52.337 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T00:59:52.337 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T00:59:52.337 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T00:59:52.337 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T00:59:52.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T00:59:52.812 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T00:59:52.854 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T00:59:52.875 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T00:59:53.722 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:19.0.0-3570.g10380055.el9.noarch 152/164 2024-05-11T00:59:53.809 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T00:59:53.855 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T00:59:53.962 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T00:59:53.995 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T00:59:54.021 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T00:59:54.021 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T00:59:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T00:59:54.147 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T00:59:54.147 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T00:59:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T00:59:54.256 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T00:59:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T00:59:54.263 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T00:59:54.467 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T00:59:54.467 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T00:59:54.551 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T00:59:54.617 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T00:59:57.233 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-test-2:19.0.0-3570.g10380055.el9.x86_64 141/164 2024-05-11T00:59:57.275 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-fuse-2:19.0.0-3570.g10380055.el9.x86_64 142/164 2024-05-11T00:59:57.324 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-nbd-2:19.0.0-3570.g10380055.el9.x86_64 143/164 2024-05-11T00:59:57.355 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T00:59:57.382 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T00:59:57.582 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-shell-2:19.0.0-3570.g10380055.el9.x86_64 146/164 2024-05-11T00:59:57.629 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T00:59:57.659 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs-devel-2:19.0.0-3570.g10380055.el9.x86_ 148/164 2024-05-11T00:59:57.689 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T00:59:57.743 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T00:59:58.343 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T00:59:58.382 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T00:59:58.403 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T00:59:59.309 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-top-2:19.0.0-3570.g10380055.el9.noarch 152/164 2024-05-11T00:59:59.404 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T00:59:59.455 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T00:59:59.551 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T00:59:59.582 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T00:59:59.609 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T00:59:59.609 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T00:59:59.701 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T00:59:59.734 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T00:59:59.734 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T00:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T00:59:59.844 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T00:59:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T00:59:59.849 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T01:00:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T01:00:00.037 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T01:00:00.066 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T01:00:00.066 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.0.0-3570.g10380055.el9.x86_64 1/164 2024-05-11T01:00:00.066 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.0.0-3570.g10380055.el9.x86_64 2/164 2024-05-11T01:00:00.066 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 3/164 2024-05-11T01:00:00.066 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.0.0-3570.g10380055.el9.x86_64 4/164 2024-05-11T01:00:00.066 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3570.g10380 5/164 2024-05-11T01:00:00.066 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.0.0-3570.g10380055.el9.x86_64 6/164 2024-05-11T01:00:00.066 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.0.0-3570.g10380055.el9.x86_64 7/164 2024-05-11T01:00:00.066 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.0.0-3570.g10380055.el9.x86_64 8/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.0.0-3570.g10380055.el9.x86_64 9/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.0.0-3570.g10380055.el9.x86_64 10/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.0.0-3570.g10380055.el9.x86_64 11/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.0.0-3570.g10380055.el9.x86_64 12/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:19.0.0-3570.g10380055.el9.x86_64 13/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-shell-2:19.0.0-3570.g10380055.el9.x86_64 14/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.0.0-3570.g10380055.el9.x86_ 15/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.0.0-3570.g10380055.el9.x86_64 16/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.0.0-3570.g10380055.el9.x86_64 17/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.0.0-3570.g10380055.el9.x86_6 18/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.0.0-3570.g10380055.el9.x86 19/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.0.0-3570.g10380055.el9.x86_64 20/164 2024-05-11T01:00:00.067 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3570.g10380055.el 21/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.0.0-3570.g10380055.el9. 22/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.0.0-3570.g10380055.el9.x86_6 23/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.0.0-3570.g10380055.el9.x86_64 24/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.0.0-3570.g10380055.el9.x86_64 25/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.0.0-3570.g10380055.el9.x86_64 26/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.0.0-3570.g10380055.el9.x86_64 27/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.0.0-3570.g10380055.el9.x86_64 28/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.0.0-3570.g10380055.el9.x86_64 29/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3570.g10380055. 30/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3570.g10380055.el9.noa 31/164 2024-05-11T01:00:00.068 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3570.g10380055.el9.n 32/164 2024-05-11T01:00:00.080 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3570.g103 33/164 2024-05-11T01:00:00.080 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3570.g10380055.el 34/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3570.g10380055.el9.noarch 35/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3570.g10380055.e 36/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.0.0-3570.g10380055.el9.noarch 37/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.0.0-3570.g10380055.el9.noarch 38/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:19.0.0-3570.g10380055.el9.noarch 39/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T01:00:00.081 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T01:00:00.082 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T01:00:00.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T01:00:00.084 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T01:00:00.085 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T01:00:00.086 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T01:00:00.087 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T01:00:00.088 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.0.0-3570.g10380055.el9.x86_64 149/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.0.0-3570.g10380055.el9.x86_64 151/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T01:00:00.089 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T01:00:00.090 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T01:00:00.090 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T01:00:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T01:00:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T01:00:03.002 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T01:00:03.002 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T01:00:03.003 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T01:00:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T01:00:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T01:00:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T01:00:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T01:00:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T01:00:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T01:00:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T01:00:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T01:00:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T01:00:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T01:00:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T01:00:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T01:00:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:03.014 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-11T01:00:03.423 DEBUG:teuthology.parallel:result is None 2024-05-11T01:00:05.696 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T01:00:05.696 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-2:19.0.0-3570.g10380055.el9.x86_64 1/164 2024-05-11T01:00:05.696 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-base-2:19.0.0-3570.g10380055.el9.x86_64 2/164 2024-05-11T01:00:05.696 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 3/164 2024-05-11T01:00:05.696 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-fuse-2:19.0.0-3570.g10380055.el9.x86_64 4/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3570.g10380 5/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mds-2:19.0.0-3570.g10380055.el9.x86_64 6/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-2:19.0.0-3570.g10380055.el9.x86_64 7/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mon-2:19.0.0-3570.g10380055.el9.x86_64 8/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-osd-2:19.0.0-3570.g10380055.el9.x86_64 9/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-radosgw-2:19.0.0-3570.g10380055.el9.x86_64 10/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-selinux-2:19.0.0-3570.g10380055.el9.x86_64 11/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-test-2:19.0.0-3570.g10380055.el9.x86_64 12/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-mirror-2:19.0.0-3570.g10380055.el9.x86_64 13/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-shell-2:19.0.0-3570.g10380055.el9.x86_64 14/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs-devel-2:19.0.0-3570.g10380055.el9.x86_ 15/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs2-2:19.0.0-3570.g10380055.el9.x86_64 16/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephsqlite-2:19.0.0-3570.g10380055.el9.x86_64 17/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados-devel-2:19.0.0-3570.g10380055.el9.x86_6 18/164 2024-05-11T01:00:05.697 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libradosstriper1-2:19.0.0-3570.g10380055.el9.x86 19/164 2024-05-11T01:00:05.698 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librgw2-2:19.0.0-3570.g10380055.el9.x86_64 20/164 2024-05-11T01:00:05.698 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3570.g10380055.el 21/164 2024-05-11T01:00:05.698 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-common-2:19.0.0-3570.g10380055.el9. 22/164 2024-05-11T01:00:05.698 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cephfs-2:19.0.0-3570.g10380055.el9.x86_6 23/164 2024-05-11T01:00:05.698 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rados-2:19.0.0-3570.g10380055.el9.x86_64 24/164 2024-05-11T01:00:05.698 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rbd-2:19.0.0-3570.g10380055.el9.x86_64 25/164 2024-05-11T01:00:05.698 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rgw-2:19.0.0-3570.g10380055.el9.x86_64 26/164 2024-05-11T01:00:05.698 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-fuse-2:19.0.0-3570.g10380055.el9.x86_64 27/164 2024-05-11T01:00:05.698 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-mirror-2:19.0.0-3570.g10380055.el9.x86_64 28/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-nbd-2:19.0.0-3570.g10380055.el9.x86_64 29/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3570.g10380055. 30/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3570.g10380055.el9.noa 31/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3570.g10380055.el9.n 32/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3570.g103 33/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3570.g10380055.el 34/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3570.g10380055.el9.noarch 35/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3570.g10380055.e 36/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-volume-2:19.0.0-3570.g10380055.el9.noarch 37/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephadm-2:19.0.0-3570.g10380055.el9.noarch 38/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-top-2:19.0.0-3570.g10380055.el9.noarch 39/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T01:00:05.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T01:00:05.700 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T01:00:05.701 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T01:00:05.702 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T01:00:05.702 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T01:00:05.702 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T01:00:05.702 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T01:00:05.702 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T01:00:05.702 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T01:00:05.703 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T01:00:05.704 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T01:00:05.705 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T01:00:05.706 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:19.0.0-3570.g10380055.el9.x86_64 149/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T01:00:05.707 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:19.0.0-3570.g10380055.el9.x86_64 151/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T01:00:05.708 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T01:00:08.420 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T01:00:08.420 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:08.420 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-05-11T01:00:08.420 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T01:00:08.420 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T01:00:08.420 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T01:00:08.421 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T01:00:08.421 INFO:teuthology.orchestra.run.smithi052.stdout: librados2-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.421 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.421 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T01:00:08.421 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T01:00:08.421 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-05-11T01:00:08.421 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T01:00:08.421 INFO:teuthology.orchestra.run.smithi052.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T01:00:08.421 INFO:teuthology.orchestra.run.smithi052.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T01:00:08.422 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T01:00:08.422 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.422 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.422 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.422 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.422 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.422 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.422 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.422 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.423 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top-2:19.0.0-3570.g10380055.el9.noarch 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T01:00:08.424 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T01:00:08.425 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T01:00:08.425 INFO:teuthology.orchestra.run.smithi052.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T01:00:08.425 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T01:00:08.425 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T01:00:08.425 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T01:00:08.425 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T01:00:08.425 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.426 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T01:00:08.427 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T01:00:08.428 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T01:00:08.428 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T01:00:08.428 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T01:00:08.428 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T01:00:08.428 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T01:00:08.428 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T01:00:08.428 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T01:00:08.428 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T01:00:08.428 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T01:00:08.429 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T01:00:08.430 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T01:00:08.431 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T01:00:08.432 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.433 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.434 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd-2:19.0.0-3570.g10380055.el9.x86_64 2024-05-11T01:00:08.434 INFO:teuthology.orchestra.run.smithi052.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T01:00:08.434 INFO:teuthology.orchestra.run.smithi052.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T01:00:08.434 INFO:teuthology.orchestra.run.smithi052.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T01:00:08.434 INFO:teuthology.orchestra.run.smithi052.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T01:00:08.434 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T01:00:08.434 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T01:00:08.434 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T01:00:08.434 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-05-11T01:00:08.859 DEBUG:teuthology.parallel:result is None 2024-05-11T01:00:08.860 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=10380055a1fc5e7d3b37b252af7045f5c2c8ba29 2024-05-11T01:00:09.043 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T01:00:09.067 INFO:teuthology.orchestra.run.smithi052.stdout:19.0.0-3570.g10380055.el9 2024-05-11T01:00:09.067 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3570.g10380055.el9 2024-05-11T01:00:09.067 INFO:teuthology.task.install:The correct ceph version 19.0.0-3570.g10380055 is installed. 2024-05-11T01:00:09.069 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=10380055a1fc5e7d3b37b252af7045f5c2c8ba29 2024-05-11T01:00:09.345 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T01:00:09.374 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-3570.g10380055.el9 2024-05-11T01:00:09.374 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3570.g10380055.el9 2024-05-11T01:00:09.374 INFO:teuthology.task.install:The correct ceph version 19.0.0-3570.g10380055 is installed. 2024-05-11T01:00:09.377 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-11T01:00:09.378 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:09.378 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T01:00:09.412 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:09.412 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T01:00:09.446 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-11T01:00:09.447 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:09.447 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T01:00:09.483 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T01:00:09.552 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:09.552 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T01:00:09.581 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T01:00:09.651 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-11T01:00:09.651 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:09.651 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T01:00:09.680 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T01:00:09.748 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:09.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T01:00:09.777 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T01:00:09.846 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-11T01:00:09.846 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:09.846 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T01:00:09.874 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T01:00:09.942 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:09.942 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T01:00:09.969 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T01:00:10.036 INFO:teuthology.run_tasks:Running task ceph... 2024-05-11T01:00:10.217 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-11T01:00:10.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /var/log/ceph 2024-05-11T01:00:10.220 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2024-05-11T01:00:10.248 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-11T01:00:10.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-11T01:00:10.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-11T01:00:10.317 INFO:tasks.ceph:Creating extra log directories... 2024-05-11T01:00:10.318 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-11T01:00:10.360 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-11T01:00:10.392 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-11T01:00:10.392 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-11T01:00:10.394 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-11T01:00:10.394 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:10.394 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-11T01:00:10.443 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-11T01:00:10.557 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-11T01:00:10.584 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:10.584 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-11T01:00:10.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-11T01:00:10.711 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-11T01:00:10.742 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-11T01:00:10.742 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-11T01:00:10.742 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038', 'branch': 'wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/snap-schedule}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7702038', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds', 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3672, 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi052.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNPzuCqcuBbnXZ3bZTAStOBBelgDv+9sQKjLwg+nqyO++1tBJi4TrMejaJOMWTNX7ndUGjaVSrhLo+v9nTXsVkQ=', 'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOjac69GAdpZVi+7u90HWwTwSnpInKp6GsFxkoCpGY07c3VhdYd+sDz2vnYoZ+hNKkf3QRjP23ZX33X8quJrn6Q='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '10380055a1fc5e7d3b37b252af7045f5c2c8ba29', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-11_00:39:26', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-11T01:00:10.743 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-11T01:00:10.745 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-11T01:00:10.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m0777 -- /var/run/ceph 2024-05-11T01:00:10.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2024-05-11T01:00:10.858 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:10.858 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T01:00:10.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T01:00:10.884 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-05-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-11 01:00:00.550849398 +0000 2024-05-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-11 00:58:03.051424826 +0000 2024-05-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-11 00:58:03.051424826 +0000 2024-05-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-05-11 00:58:03.051424826 +0000 2024-05-11T01:00:10.941 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T01:00:11.004 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-11T01:00:11.004 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-11T01:00:11.004 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000104287 s, 4.9 MB/s 2024-05-11T01:00:11.006 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T01:00:11.063 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-05-11T01:00:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T01:00:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T01:00:11.119 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-11T01:00:11.119 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T01:00:11.119 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T01:00:11.119 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-11 01:00:00.550849398 +0000 2024-05-11T01:00:11.119 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-11 00:58:03.391420410 +0000 2024-05-11T01:00:11.119 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-11 00:58:03.391420410 +0000 2024-05-11T01:00:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-05-11 00:58:03.391420410 +0000 2024-05-11T01:00:11.120 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T01:00:11.183 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-11T01:00:11.183 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-11T01:00:11.183 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 8.4118e-05 s, 6.1 MB/s 2024-05-11T01:00:11.184 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T01:00:11.241 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-05-11T01:00:11.298 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T01:00:11.298 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T01:00:11.298 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-11T01:00:11.298 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T01:00:11.298 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T01:00:11.298 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-11 01:00:00.550849398 +0000 2024-05-11T01:00:11.299 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-11 00:58:03.741415865 +0000 2024-05-11T01:00:11.299 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-11 00:58:03.741415865 +0000 2024-05-11T01:00:11.299 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-05-11 00:58:03.741415865 +0000 2024-05-11T01:00:11.299 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T01:00:11.363 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-11T01:00:11.363 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-11T01:00:11.363 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 8.5639e-05 s, 6.0 MB/s 2024-05-11T01:00:11.364 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T01:00:11.421 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-05-11T01:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T01:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T01:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-11T01:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T01:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T01:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-11 01:00:00.551849385 +0000 2024-05-11T01:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-11 00:58:04.085411397 +0000 2024-05-11T01:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-11 00:58:04.085411397 +0000 2024-05-11T01:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-05-11 00:58:04.085411397 +0000 2024-05-11T01:00:11.479 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T01:00:11.542 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-11T01:00:11.542 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-11T01:00:11.542 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000101847 s, 5.0 MB/s 2024-05-11T01:00:11.543 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T01:00:11.601 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:11.601 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:11.601 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T01:00:11.618 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T01:00:11.619 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-05-11T01:00:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T01:00:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T01:00:11.675 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-11T01:00:11.675 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T01:00:11.675 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T01:00:11.675 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-05-11 00:59:54.952945239 +0000 2024-05-11T01:00:11.675 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-05-11 00:58:03.052154054 +0000 2024-05-11T01:00:11.675 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-05-11 00:58:03.052154054 +0000 2024-05-11T01:00:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-05-11 00:58:03.052154054 +0000 2024-05-11T01:00:11.676 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T01:00:11.739 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-05-11T01:00:11.739 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-05-11T01:00:11.739 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000207855 s, 2.5 MB/s 2024-05-11T01:00:11.740 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T01:00:11.798 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-05-11T01:00:11.853 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T01:00:11.853 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T01:00:11.854 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-11T01:00:11.854 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T01:00:11.854 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T01:00:11.854 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-05-11 00:59:54.952945239 +0000 2024-05-11T01:00:11.854 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-05-11 00:58:03.386145504 +0000 2024-05-11T01:00:11.854 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-05-11 00:58:03.386145504 +0000 2024-05-11T01:00:11.854 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-05-11 00:58:03.386145504 +0000 2024-05-11T01:00:11.854 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T01:00:11.917 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-05-11T01:00:11.917 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-05-11T01:00:11.917 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000188191 s, 2.7 MB/s 2024-05-11T01:00:11.919 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T01:00:11.976 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-05-11T01:00:12.034 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T01:00:12.034 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T01:00:12.034 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-11T01:00:12.035 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T01:00:12.035 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T01:00:12.035 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-05-11 00:59:54.953945214 +0000 2024-05-11T01:00:12.035 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-05-11 00:58:03.736136544 +0000 2024-05-11T01:00:12.035 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-05-11 00:58:03.736136544 +0000 2024-05-11T01:00:12.035 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-05-11 00:58:03.736136544 +0000 2024-05-11T01:00:12.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T01:00:12.099 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-05-11T01:00:12.099 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-05-11T01:00:12.099 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000186234 s, 2.7 MB/s 2024-05-11T01:00:12.100 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T01:00:12.156 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-05-11T01:00:12.211 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T01:00:12.211 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T01:00:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-11T01:00:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T01:00:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T01:00:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-05-11 00:59:54.953945214 +0000 2024-05-11T01:00:12.211 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-05-11 00:58:04.076127840 +0000 2024-05-11T01:00:12.212 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-05-11 00:58:04.076127840 +0000 2024-05-11T01:00:12.212 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-05-11 00:58:04.076127840 +0000 2024-05-11T01:00:12.212 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T01:00:12.274 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-05-11T01:00:12.274 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-05-11T01:00:12.274 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00021152 s, 2.4 MB/s 2024-05-11T01:00:12.275 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T01:00:12.332 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:12.332 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi052.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-11T01:00:12.332 INFO:tasks.ceph:Generating config... 2024-05-11T01:00:12.343 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-11T01:00:12.343 INFO:tasks.ceph:[client] debug client = 20 2024-05-11T01:00:12.343 INFO:tasks.ceph:[client] debug ms = 1 2024-05-11T01:00:12.343 INFO:tasks.ceph:[client] pid file = 2024-05-11T01:00:12.343 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-11T01:00:12.343 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-11T01:00:12.344 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-11T01:00:12.345 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-11T01:00:12.345 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-11T01:00:12.345 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-11T01:00:12.345 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-11T01:00:12.345 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-11T01:00:12.345 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-11T01:00:12.346 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-11T01:00:12.346 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-11T01:00:12.346 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-11T01:00:12.346 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-11T01:00:12.346 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-11T01:00:12.346 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-11T01:00:12.346 INFO:tasks.ceph:Setting up mon.a... 2024-05-11T01:00:12.346 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-11T01:00:12.402 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.keyring 2024-05-11T01:00:12.405 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-11T01:00:12.453 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T01:00:12.480 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.52'), ('mon.b', '172.21.15.119'), ('mon.c', '[v2:172.21.15.119:3301,v1:172.21.15.119:6790]')] 2024-05-11T01:00:12.481 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.52,172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-11T01:00:12.482 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:12.482 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-11T01:00:12.538 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.52 --add b 172.21.15.119 --addv c '[v2:172.21.15.119:3301,v1:172.21.15.119:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool: generated fsid f8411cb1-0508-49cc-906f-2364b6835afe 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:setting min_mon_release = quincy 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 0 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:fsid f8411cb1-0508-49cc-906f-2364b6835afe 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:last_changed 2024-05-11T01:00:12.627945+0000 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-05-11T01:00:12.627945+0000 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:min_mon_release 17 (quincy) 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:election_strategy: 1 2024-05-11T01:00:12.627 INFO:teuthology.orchestra.run.smithi052.stdout:0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-05-11T01:00:12.628 INFO:teuthology.orchestra.run.smithi052.stdout:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-05-11T01:00:12.628 INFO:teuthology.orchestra.run.smithi052.stdout:2: [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] mon.c 2024-05-11T01:00:12.628 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-11T01:00:12.629 DEBUG:teuthology.orchestra.run.smithi052:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-11T01:00:12.645 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f8411cb1-0508-49cc-906f-2364b6835afe... 2024-05-11T01:00:12.646 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T01:00:12.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T01:00:12.731 INFO:teuthology.orchestra.run.smithi052.stdout:[global] 2024-05-11T01:00:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: chdir = "" 2024-05-11T01:00:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T01:00:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: auth supported = cephx 2024-05-11T01:00:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: filestore xattr use omap = true 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: mon clock drift allowed = 1.000 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: osd crush chooseleaf type = 0 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: auth debug = true 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on old message = true 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on bug = true 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: mon pg warn max object skew = 0 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T01:00:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: osd pool default size = 2 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd allow primary affinity = true 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd allow pg remap = true 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on legacy crush tunables = false 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on crush straw calc version zero = false 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on no sortbitwise = false 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on osd down out interval zero = false 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on too few osds = false 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: mon_allow_pool_size_one = true 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T01:00:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: osd default data pool replay window = 5 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: mon allow pool delete = true 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: mon cluster log file level = debug 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: debug asserts on shutdown = true 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: mon health detail to clog = false 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: mon host = "172.21.15.52,172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790]" 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: fsid = f8411cb1-0508-49cc-906f-2364b6835afe 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout:[osd] 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: osd journal size = 100 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: osd scrub load threshold = 5.0 2024-05-11T01:00:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: osd scrub max interval = 600 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd mclock profile = high_recovery_ops 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd recover clone overlap = true 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd recovery max chunk = 1048576 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug shutdown = true 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug op order = true 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug verify stray on activate = true 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug trim objects = true 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd open classes on start = true 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug pg log writeout = true 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd deep scrub update digest min age = 30 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: osd map max advance = 10 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: journal zero on create = true 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: filestore ondisk finisher threads = 3 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: filestore apply finisher threads = 3 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: bdev debug aio = true 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug misdirected ops = true 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: bdev async discard = True 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: bdev enable discard = True 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore allocator = bitmap 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore block size = 96636764160 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore fsck on mount = True 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: debug bluefs = 1/20 2024-05-11T01:00:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: debug bluestore = 1/20 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: debug osd = 20 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: debug rocksdb = 4/10 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd full ratio = 0.9 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd nearfull ratio = 0.8 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: osd failsafe full ratio = 0.95 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: osd objectstore = bluestore 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: osd op complaint time = 180 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr] 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: debug mgr = 20 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: debug mon = 20 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: debug auth = 20 2024-05-11T01:00:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min pgs per osd = 4 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min bytes per osd = 10 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: mgr/telemetry/nag = false 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: debug client = 20 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: debug finisher = 20 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mon] 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: debug mon = 20 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: debug paxos = 20 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: debug auth = 20 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: mon data avail warn = 5 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: mon mgr mkfs grace = 240 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min pgs per osd = 4 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd reporter subtree level = osd 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd prime pg temp = true 2024-05-11T01:00:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min bytes per osd = 10 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: # 1m isn't quite enough 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: mon_down_mkfs_grace = 2m 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_filestore_osds = false 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: mon down mkfs grace = 300 2024-05-11T01:00:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: mon op complaint time = 120 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout:[client] 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: rgw cache enabled = true 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: rgw enable ops log = true 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: rgw enable usage log = true 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: client mount timeout = 600 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: debug client = 20 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: pid file = "" 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: rados mon op timeout = 900 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: rados osd op timeout = 900 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.a] 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.b] 2024-05-11T01:00:12.740 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.c] 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a] 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c] 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b] 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d] 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout:[mds] 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: debug mds = 20 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: debug mds balancer = 20 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: mds debug frag = True 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: mds debug scatterstat = True 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: mds op complaint time = 180 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: mds verify scatter = True 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: osd op complaint time = 180 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: rados mon op timeout = 900 2024-05-11T01:00:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: rados osd op timeout = 900 2024-05-11T01:00:12.742 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-05-11T01:00:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-05-11T01:00:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T01:00:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-05-11T01:00:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-05-11T01:00:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.52,172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790]" 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = f8411cb1-0508-49cc-906f-2364b6835afe 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-05-11T01:00:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-05-11T01:00:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: debug finisher = 20 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-05-11T01:00:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: mon down mkfs grace = 300 2024-05-11T01:00:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-05-11T01:00:12.751 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-05-11T01:00:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-05-11T01:00:12.754 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-11T01:00:12.754 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-11T01:00:12.850 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-11T01:00:12.850 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:12.850 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-11T01:00:12.866 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:12.866 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-11T01:00:12.922 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:00:12.922 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:12.922 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-11T01:00:12.922 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T01:00:12.998 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:12.998 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:13.054 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:00:13.054 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:13.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-11T01:00:13.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T01:00:13.088 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:13.089 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:13.144 INFO:tasks.ceph:Setting up mon nodes... 2024-05-11T01:00:13.144 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-11T01:00:13.144 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-11T01:00:13.198 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-11T01:00:13.200 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-11T01:00:13.258 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-11T01:00:13.261 INFO:tasks.ceph:Setting up mds nodes... 2024-05-11T01:00:13.261 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-11T01:00:13.314 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-11T01:00:13.317 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-11T01:00:13.342 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-11T01:00:13.436 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-11T01:00:13.438 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-11T01:00:13.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-11T01:00:13.515 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-11T01:00:13.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-11T01:00:13.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-11T01:00:13.637 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-11T01:00:13.639 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-11T01:00:13.664 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-11T01:00:13.664 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-11T01:00:13.707 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-11T01:00:13.720 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-11T01:00:13.802 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-11T01:00:13.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-11T01:00:13.856 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-11T01:00:13.869 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-11T01:00:13.950 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-11T01:00:13.962 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-11T01:00:13.963 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi052.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-11T01:00:13.963 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-11T01:00:13.989 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:13.989 INFO:tasks.ceph:role: osd.0 2024-05-11T01:00:13.989 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:00:13.990 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-11T01:00:14.054 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T01:00:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T01:00:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T01:00:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T01:00:14.055 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T01:00:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-05-11T01:00:14.055 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T01:00:14.055 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T01:00:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T01:00:14.055 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T01:00:14.087 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-05-11T01:00:14.089 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-05-11T01:00:14.089 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-11T01:00:14.159 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-11T01:00:14.229 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-11T01:00:14.296 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:14.296 INFO:tasks.ceph:role: osd.1 2024-05-11T01:00:14.296 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:00:14.297 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-11T01:00:14.363 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T01:00:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T01:00:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T01:00:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T01:00:14.363 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T01:00:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-05-11T01:00:14.363 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T01:00:14.363 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T01:00:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T01:00:14.364 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T01:00:14.396 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-05-11T01:00:14.398 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-05-11T01:00:14.398 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-11T01:00:14.468 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-11T01:00:14.538 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-11T01:00:14.606 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:14.607 INFO:tasks.ceph:role: osd.2 2024-05-11T01:00:14.607 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:00:14.607 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-11T01:00:14.673 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T01:00:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T01:00:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T01:00:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T01:00:14.673 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T01:00:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-05-11T01:00:14.673 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T01:00:14.673 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T01:00:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T01:00:14.674 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T01:00:14.705 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-05-11T01:00:14.707 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-05-11T01:00:14.707 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-11T01:00:14.778 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-11T01:00:14.852 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-11T01:00:14.924 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:14.924 INFO:tasks.ceph:role: osd.3 2024-05-11T01:00:14.924 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:00:14.924 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-11T01:00:14.989 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T01:00:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T01:00:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T01:00:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T01:00:14.990 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T01:00:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-05-11T01:00:14.990 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T01:00:14.990 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T01:00:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T01:00:14.990 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T01:00:15.022 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-05-11T01:00:15.024 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-05-11T01:00:15.024 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-11T01:00:15.094 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-11T01:00:15.165 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:15.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:15.262+0000 7f34d5a0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-11T01:00:15.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:15.262+0000 7f34d5a0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-11T01:00:15.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:15.262+0000 7f34d5a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T01:00:15.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:15.262+0000 7f34d5a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T01:00:15.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:15.262+0000 7f34d5a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T01:00:15.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:15.262+0000 7f34d5a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-11T01:00:18.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-11T01:00:18.148 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:18.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:18.241+0000 7f3948e12640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-11T01:00:18.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:18.241+0000 7f3948e12640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-11T01:00:18.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:18.241+0000 7f3948e12640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T01:00:18.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:18.241+0000 7f3948e12640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T01:00:18.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:18.241+0000 7f3948e12640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T01:00:18.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:18.241+0000 7f3948e12640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-11T01:00:21.086 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-11T01:00:21.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:21.210+0000 7f59e3e3f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-11T01:00:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:21.211+0000 7f59e3e3f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-11T01:00:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:21.211+0000 7f59e3e3f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T01:00:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:21.211+0000 7f59e3e3f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T01:00:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:21.211+0000 7f59e3e3f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T01:00:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:21.211+0000 7f59e3e3f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-11T01:00:24.057 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-11T01:00:24.088 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:24.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:24.180+0000 7f5e18150640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-11T01:00:24.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:24.181+0000 7f5e18150640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-11T01:00:24.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:24.181+0000 7f5e18150640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T01:00:24.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:24.181+0000 7f5e18150640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T01:00:24.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:24.181+0000 7f5e18150640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T01:00:24.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:24.181+0000 7f5e18150640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-11T01:00:27.028 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-11T01:00:27.059 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-11T01:00:27.089 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:27.089 INFO:tasks.ceph:role: osd.4 2024-05-11T01:00:27.090 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:00:27.090 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-11T01:00:27.166 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T01:00:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T01:00:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T01:00:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T01:00:27.167 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T01:00:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-05-11T01:00:27.167 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T01:00:27.167 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T01:00:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T01:00:27.167 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T01:00:27.669 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-05-11T01:00:27.671 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-05-11T01:00:27.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-11T01:00:27.791 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-11T01:00:27.823 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-11T01:00:27.892 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:27.892 INFO:tasks.ceph:role: osd.5 2024-05-11T01:00:27.892 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:00:27.892 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-11T01:00:27.964 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T01:00:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T01:00:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T01:00:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T01:00:27.964 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T01:00:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-05-11T01:00:27.964 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T01:00:27.964 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T01:00:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T01:00:27.965 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T01:00:28.424 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-05-11T01:00:28.426 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-05-11T01:00:28.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-11T01:00:28.508 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-11T01:00:28.578 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-11T01:00:28.645 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:28.646 INFO:tasks.ceph:role: osd.6 2024-05-11T01:00:28.646 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:00:28.646 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-11T01:00:28.718 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T01:00:28.718 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T01:00:28.718 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T01:00:28.718 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T01:00:28.718 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T01:00:28.719 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-05-11T01:00:28.719 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T01:00:28.719 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T01:00:28.719 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T01:00:28.719 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T01:00:29.243 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-05-11T01:00:29.245 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-05-11T01:00:29.246 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-11T01:00:29.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-11T01:00:29.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-11T01:00:29.479 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T01:00:29.479 INFO:tasks.ceph:role: osd.7 2024-05-11T01:00:29.480 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:00:29.480 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-11T01:00:29.552 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T01:00:29.552 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T01:00:29.552 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T01:00:29.552 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T01:00:29.553 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T01:00:29.553 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-05-11T01:00:29.553 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T01:00:29.553 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T01:00:29.553 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T01:00:29.553 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T01:00:30.061 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-05-11T01:00:30.063 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-05-11T01:00:30.063 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-11T01:00:30.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-11T01:00:30.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:30.298+0000 7f11e0f33640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-11T01:00:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:30.299+0000 7f11e0f33640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-11T01:00:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:30.299+0000 7f11e0f33640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T01:00:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:30.299+0000 7f11e0f33640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T01:00:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:30.299+0000 7f11e0f33640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T01:00:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:30.299+0000 7f11e0f33640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-11T01:00:33.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-11T01:00:33.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:33.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:33.273+0000 7fdd665c0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-11T01:00:33.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:33.274+0000 7fdd665c0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-11T01:00:33.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:33.274+0000 7fdd665c0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T01:00:33.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:33.274+0000 7fdd665c0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T01:00:33.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:33.274+0000 7fdd665c0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T01:00:33.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:33.274+0000 7fdd665c0640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-11T01:00:36.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-11T01:00:36.166 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:36.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:36.257+0000 7fb502140640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-11T01:00:36.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:36.257+0000 7fb502140640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-11T01:00:36.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:36.257+0000 7fb502140640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T01:00:36.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:36.257+0000 7fb502140640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T01:00:36.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:36.257+0000 7fb502140640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T01:00:36.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:36.257+0000 7fb502140640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-11T01:00:39.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-11T01:00:39.144 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:39.229+0000 7f9c42461640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-11T01:00:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:39.229+0000 7f9c42461640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-11T01:00:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:39.229+0000 7f9c42461640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T01:00:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:39.229+0000 7f9c42461640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T01:00:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:39.229+0000 7f9c42461640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T01:00:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:39.229+0000 7f9c42461640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-11T01:00:40.744 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:00:40.747 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:00:42.080 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-11T01:00:42.109 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-11T01:00:42.110 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:42.110 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-11T01:00:42.146 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:42.146 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-11T01:00:42.217 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:42.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-11T01:00:42.287 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:42.287 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-11T01:00:42.356 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:42.356 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-11T01:00:42.425 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:42.425 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-11T01:00:42.493 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:42.493 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-11T01:00:42.560 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:42.560 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-11T01:00:42.590 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:42.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-11T01:00:42.659 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:42.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-11T01:00:42.727 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:42.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-11T01:00:42.794 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:42.794 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-11T01:00:42.861 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:42.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-11T01:00:42.931 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:42.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-11T01:00:42.996 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:42.997 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-11T01:00:43.016 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:43.016 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-11T01:00:43.075 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:43.076 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-11T01:00:43.092 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:43.092 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-11T01:00:43.150 INFO:tasks.ceph:Adding keys to all mons... 2024-05-11T01:00:43.150 DEBUG:teuthology.orchestra.run.smithi052:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-11T01:00:43.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-11T01:00:43.194 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr.y] 2024-05-11T01:00:43.194 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5medHbCxAASin8Gv729ZztriQmuTTTYQ== 2024-05-11T01:00:43.194 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a] 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5mP3rNEhAAP5nXdayem1rp+TLkIpVbXg== 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c] 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5mmQINGhAAyEyoarOUTjK64GfK5PZ5bQ== 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.0] 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAfwz5mJZqrDxAAq3xCv8Pf2DPOEL2Tovrbeg== 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.1] 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAiwz5mkbxtDhAAHD/TxmE4UfGOWvArcQsdNA== 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.2] 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAlwz5mgOegDBAAKU+gnyyeIwLuvXRfzOD/ZQ== 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.3] 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAowz5mERnWChAAZVaafqVtHNgrii/OZSYXdg== 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr.x] 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5mj0l4DxAAnDj4+dETpUu/ykNczo5dvA== 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b] 2024-05-11T01:00:43.195 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5m7H/HHhAAFque3IN7APQjExckFVQiQg== 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d] 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5mXGYNJhAA9loGi/oMhZT+LWktBsVhfw== 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.4] 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAuwz5mbL3bERAAO7anuNWH7NtNl5//BukL8g== 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.5] 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAxwz5myrFsEBAAvGSIqmyuovYeRCSoDNTLxg== 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.6] 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQA0wz5mythuDxAAJh/fQNHjv0kVV420jaSiyQ== 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.7] 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQA3wz5mSAXGDRAAoWVOOP2dLYGaGsue3vxWwA== 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-05-11T01:00:43.196 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5mUO0yKhAAfS2pcT6nSB+NrJccryMO8A== 2024-05-11T01:00:43.197 INFO:teuthology.orchestra.run.smithi052.stdout:[client.2] 2024-05-11T01:00:43.197 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5maFjoLxAAwD/wgFjjG9lbeRLUtV4BRQ== 2024-05-11T01:00:43.197 INFO:teuthology.orchestra.run.smithi052.stdout:[client.1] 2024-05-11T01:00:43.197 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5m3AodMxAAXArXFLqTR6kBG8KTzop93w== 2024-05-11T01:00:43.197 INFO:teuthology.orchestra.run.smithi052.stdout:[client.3] 2024-05-11T01:00:43.197 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5mEb+wOBAAI8i4rWMD8GWMUf6p0HICRA== 2024-05-11T01:00:43.215 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.y] 2024-05-11T01:00:43.215 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5medHbCxAASin8Gv729ZztriQmuTTTYQ== 2024-05-11T01:00:43.215 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5mP3rNEhAAP5nXdayem1rp+TLkIpVbXg== 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5mmQINGhAAyEyoarOUTjK64GfK5PZ5bQ== 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAfwz5mJZqrDxAAq3xCv8Pf2DPOEL2Tovrbeg== 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAiwz5mkbxtDhAAHD/TxmE4UfGOWvArcQsdNA== 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAlwz5mgOegDBAAKU+gnyyeIwLuvXRfzOD/ZQ== 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAowz5mERnWChAAZVaafqVtHNgrii/OZSYXdg== 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5mj0l4DxAAnDj4+dETpUu/ykNczo5dvA== 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5m7H/HHhAAFque3IN7APQjExckFVQiQg== 2024-05-11T01:00:43.216 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5mXGYNJhAA9loGi/oMhZT+LWktBsVhfw== 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAuwz5mbL3bERAAO7anuNWH7NtNl5//BukL8g== 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAxwz5myrFsEBAAvGSIqmyuovYeRCSoDNTLxg== 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.6] 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQA0wz5mythuDxAAJh/fQNHjv0kVV420jaSiyQ== 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.7] 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQA3wz5mSAXGDRAAoWVOOP2dLYGaGsue3vxWwA== 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5mUO0yKhAAfS2pcT6nSB+NrJccryMO8A== 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout:[client.2] 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5maFjoLxAAwD/wgFjjG9lbeRLUtV4BRQ== 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout:[client.1] 2024-05-11T01:00:43.217 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5m3AodMxAAXArXFLqTR6kBG8KTzop93w== 2024-05-11T01:00:43.218 INFO:teuthology.orchestra.run.smithi119.stdout:[client.3] 2024-05-11T01:00:43.218 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAdwz5mEb+wOBAAI8i4rWMD8GWMUf6p0HICRA== 2024-05-11T01:00:43.218 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T01:00:43.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T01:00:43.311 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T01:00:43.313 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T01:00:43.369 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T01:00:43.372 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T01:00:43.426 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.479 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.532 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.534 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.585 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.639 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T01:00:43.642 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T01:00:43.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T01:00:43.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T01:00:43.749 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T01:00:43.752 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T01:00:43.805 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.858 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.860 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.912 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.965 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:43.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T01:00:44.020 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T01:00:44.022 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T01:00:44.076 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T01:00:44.078 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T01:00:44.132 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T01:00:44.134 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T01:00:44.189 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T01:00:44.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T01:00:44.245 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-11T01:00:44.246 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-11T01:00:44.271 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T01:00:44.369 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-11T01:00:44.395 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-11T01:00:44.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T01:00:44.517 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-11T01:00:44.543 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-11T01:00:44.609 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T01:00:44.704 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-11T01:00:44.729 DEBUG:teuthology.orchestra.run.smithi052:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:44.731 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-11T01:00:44.785 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-11T01:00:44.786 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-11T01:00:44.786 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-11T01:00:44.789 INFO:tasks.ceph.mon.a:Started 2024-05-11T01:00:44.789 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-11T01:00:44.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-11T01:00:44.828 INFO:tasks.ceph.mon.b:Started 2024-05-11T01:00:44.828 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-11T01:00:44.829 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-11T01:00:44.830 INFO:tasks.ceph.mon.c:Started 2024-05-11T01:00:44.830 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-11T01:00:44.831 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-11T01:00:44.831 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-11T01:00:44.835 INFO:tasks.ceph.mgr.y:Started 2024-05-11T01:00:44.835 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-11T01:00:44.835 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-11T01:00:44.837 INFO:tasks.ceph.mgr.x:Started 2024-05-11T01:00:44.838 INFO:tasks.ceph:Setting crush tunables to default 2024-05-11T01:00:44.838 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd crush tunables default 2024-05-11T01:00:44.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.941+0000 7fede65e4640 1 Processor -- start 2024-05-11T01:00:44.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.941+0000 7fede65e4640 1 -- start start 2024-05-11T01:00:44.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.941+0000 7fede65e4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fede015f2f0 0x7fede00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:44.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.941+0000 7fede65e4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fede00d4e40 0x7fede00d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.941+0000 7fede65e4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede00d3a90 con 0x7fede00d2d60 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.941+0000 7fede65e4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fede00d3d70 con 0x7fede00d4e40 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.941+0000 7fede65e4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fede00d4050 con 0x7fede015f2f0 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddf7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fede00d4e40 0x7fede00d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddf7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fede00d4e40 0x7fede00d5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45444/0 (socket says 172.21.15.52:45444) 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddf7fe640 1 -- 172.21.15.52:0/842624676 learned_addr learned my addr 172.21.15.52:0/842624676 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddffff640 1 --2- 172.21.15.52:0/842624676 >> v2:172.21.15.119:3300/0 conn(0x7fede015f2f0 0x7fede00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddf7fe640 1 -- 172.21.15.52:0/842624676 >> v2:172.21.15.52:3300/0 conn(0x7fede00d4e40 msgr2=0x7fede00d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddf7fe640 1 -- 172.21.15.52:0/842624676 >> v2:172.21.15.52:3300/0 conn(0x7fede00d4e40 msgr2=0x7fede00d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-11T01:00:44.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddf7fe640 1 --2- 172.21.15.52:0/842624676 >> v2:172.21.15.52:3300/0 conn(0x7fede00d4e40 0x7fede00d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T01:00:44.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddf7fe640 1 --2- 172.21.15.52:0/842624676 >> v2:172.21.15.52:3300/0 conn(0x7fede00d4e40 0x7fede00d5290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:00:44.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddffff640 1 -- 172.21.15.52:0/842624676 >> v2:172.21.15.119:3300/0 conn(0x7fede015f2f0 msgr2=0x7fede00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-11T01:00:44.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddffff640 1 -- 172.21.15.52:0/842624676 >> v2:172.21.15.119:3300/0 conn(0x7fede015f2f0 msgr2=0x7fede00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-11T01:00:44.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddffff640 1 --2- 172.21.15.52:0/842624676 >> v2:172.21.15.119:3300/0 conn(0x7fede015f2f0 0x7fede00cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T01:00:44.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:44.942+0000 7feddffff640 1 --2- 172.21.15.52:0/842624676 >> v2:172.21.15.119:3300/0 conn(0x7fede015f2f0 0x7fede00cfca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:00:45.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884545817 0 0) 0x7fede00d3a90 con 0x7fede00d2d60 2024-05-11T01:00:45.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedc8003520 con 0x7fede00d2d60 2024-05-11T01:00:45.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4256721786 0 0) 0x7fedc8003520 con 0x7fede00d2d60 2024-05-11T01:00:45.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 >> v2:172.21.15.119:3300/0 conn(0x7fede015f2f0 msgr2=0x7fede00cfca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T01:00:45.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 --2- 172.21.15.52:0/842624676 >> v2:172.21.15.119:3300/0 conn(0x7fede015f2f0 0x7fede00cfca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 >> v2:172.21.15.52:3300/0 conn(0x7fede00d4e40 msgr2=0x7fede00d5290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T01:00:45.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 --2- 172.21.15.52:0/842624676 >> v2:172.21.15.52:3300/0 conn(0x7fede00d4e40 0x7fede00d5290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede00b4e60 con 0x7fede00d2d60 2024-05-11T01:00:45.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fedd8001a70 con 0x7fede00d2d60 2024-05-11T01:00:45.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.107+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 >> v1:172.21.15.119:6789/0 conn(0x7fede00d2d60 legacy=0x7fede00d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7feddeffd640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc8003aa0 0x7fedc8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7feddeffd640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fedc8004800 0x7fedc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7feddeffd640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fedc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fede00d3d70 con 0x7fedc8005210 2024-05-11T01:00:45.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fede00d4050 con 0x7fedc8003aa0 2024-05-11T01:00:45.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fedc800c1c0 con 0x7fedc8004800 2024-05-11T01:00:45.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7feddf7fe640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fedc8004800 0x7fedc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7fede4b81640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fedc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7feddffff640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc8003aa0 0x7fedc8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7fede4b81640 1 -- 172.21.15.52:0/842624676 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fedc8004800 msgr2=0x7fedc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:45.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7fede4b81640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fedc8004800 0x7fedc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7fede4b81640 1 -- 172.21.15.52:0/842624676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc8003aa0 msgr2=0x7fedc8003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:45.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7fede4b81640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedc8003aa0 0x7fedc8003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.108+0000 7fede4b81640 1 -- 172.21.15.52:0/842624676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc8003520 con 0x7fedc8005210 2024-05-11T01:00:45.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.109+0000 7fede4b81640 1 -- 172.21.15.52:0/842624676 reap_dead start 2024-05-11T01:00:45.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.109+0000 7fede4b81640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fedc800b9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fedd800c4c0 tx=0x7fedc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7695150a10ddf4c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.110+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8033070 con 0x7fedc8005210 2024-05-11T01:00:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.110+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd802f650 con 0x7fedc8005210 2024-05-11T01:00:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.110+0000 7feddeffd640 1 -- 172.21.15.52:0/842624676 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd800c0e0 con 0x7fedc8005210 2024-05-11T01:00:45.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.111+0000 7fede65e4640 1 -- 172.21.15.52:0/842624676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 msgr2=0x7fedc800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.111+0000 7fede65e4640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fedc800b9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fedd800c4c0 tx=0x7fedc8005830 comp rx=0 tx=0).stop 2024-05-11T01:00:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.111+0000 7fede65e4640 1 -- 172.21.15.52:0/842624676 shutdown_connections 2024-05-11T01:00:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.111+0000 7fede65e4640 1 --2- 172.21.15.52:0/842624676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fedc800b9c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.111+0000 7fede65e4640 1 -- 172.21.15.52:0/842624676 >> 172.21.15.52:0/842624676 conn(0x7fede008ebb0 msgr2=0x7fede009e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.111+0000 7fede65e4640 1 -- 172.21.15.52:0/842624676 shutdown_connections 2024-05-11T01:00:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.111+0000 7fede65e4640 1 -- 172.21.15.52:0/842624676 wait complete. 2024-05-11T01:00:45.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.113+0000 7fede65e4640 1 Processor -- start 2024-05-11T01:00:45.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.113+0000 7fede65e4640 1 -- start start 2024-05-11T01:00:45.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7fede65e4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fede00a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7fede65e4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fede00a2c00 0x7fede00cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7fede65e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede015a210 0x7fede00cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7fede65e4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fede015f850 con 0x7fedc8005210 2024-05-11T01:00:45.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7fede65e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fede015f6d0 con 0x7fede015a210 2024-05-11T01:00:45.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7fede65e4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fede00d5290 con 0x7fede00a2c00 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fede00a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fede00a26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45482/0 (socket says 172.21.15.52:45482) 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddffff640 1 -- 172.21.15.52:0/3258692692 learned_addr learned my addr 172.21.15.52:0/3258692692 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7fede4b81640 1 --2- 172.21.15.52:0/3258692692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede015a210 0x7fede00cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddf7fe640 1 --2- 172.21.15.52:0/3258692692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fede00a2c00 0x7fede00cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddffff640 1 -- 172.21.15.52:0/3258692692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fede00a2c00 msgr2=0x7fede00cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddffff640 1 --2- 172.21.15.52:0/3258692692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fede00a2c00 0x7fede00cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddffff640 1 -- 172.21.15.52:0/3258692692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede015a210 msgr2=0x7fede00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddffff640 1 --2- 172.21.15.52:0/3258692692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede015a210 0x7fede00cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddffff640 1 -- 172.21.15.52:0/3258692692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede009f220 con 0x7fedc8005210 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.114+0000 7feddffff640 1 --2- 172.21.15.52:0/3258692692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fede00a26c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fedd00108f0 tx=0x7fedd0010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.115+0000 7feddcff9640 1 -- 172.21.15.52:0/3258692692 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd001a020 con 0x7fedc8005210 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.115+0000 7feddcff9640 1 -- 172.21.15.52:0/3258692692 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd000e9c0 con 0x7fedc8005210 2024-05-11T01:00:45.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.115+0000 7fede65e4640 1 -- 172.21.15.52:0/3258692692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede009f500 con 0x7fedc8005210 2024-05-11T01:00:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.115+0000 7feddcff9640 1 -- 172.21.15.52:0/3258692692 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd000edb0 con 0x7fedc8005210 2024-05-11T01:00:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.115+0000 7fede65e4640 1 -- 172.21.15.52:0/3258692692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede009fac0 con 0x7fedc8005210 2024-05-11T01:00:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.115+0000 7feddcff9640 1 -- 172.21.15.52:0/3258692692 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fedd001d020 con 0x7fedc8005210 2024-05-11T01:00:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.115+0000 7feddcff9640 1 -- 172.21.15.52:0/3258692692 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fedd0014870 con 0x7fedc8005210 2024-05-11T01:00:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.116+0000 7fede65e4640 1 -- 172.21.15.52:0/3258692692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede00695d0 con 0x7fedc8005210 2024-05-11T01:00:45.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.118+0000 7feddcff9640 1 -- 172.21.15.52:0/3258692692 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fedd0024020 con 0x7fedc8005210 2024-05-11T01:00:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.186+0000 7fede65e4640 1 -- 172.21.15.52:0/3258692692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fede015ac80 con 0x7fedc8005210 2024-05-11T01:00:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.187+0000 7feddcff9640 1 -- 172.21.15.52:0/3258692692 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fedd0014ac0 con 0x7fedc8005210 2024-05-11T01:00:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:adjusted tunables profile to default 2024-05-11T01:00:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.188+0000 7fede65e4640 1 -- 172.21.15.52:0/3258692692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 msgr2=0x7fede00a26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.188+0000 7fede65e4640 1 --2- 172.21.15.52:0/3258692692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fede00a26c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fedd00108f0 tx=0x7fedd0010db0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.189+0000 7fede65e4640 1 -- 172.21.15.52:0/3258692692 shutdown_connections 2024-05-11T01:00:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.189+0000 7fede65e4640 1 --2- 172.21.15.52:0/3258692692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fede015a210 0x7fede00cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.189+0000 7fede65e4640 1 --2- 172.21.15.52:0/3258692692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fede00a2c00 0x7fede00cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.189+0000 7fede65e4640 1 --2- 172.21.15.52:0/3258692692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedc8005210 0x7fede00a26c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.189+0000 7fede65e4640 1 -- 172.21.15.52:0/3258692692 >> 172.21.15.52:0/3258692692 conn(0x7fede008ebb0 msgr2=0x7fede006dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.189+0000 7fede65e4640 1 -- 172.21.15.52:0/3258692692 shutdown_connections 2024-05-11T01:00:45.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:45.189+0000 7fede65e4640 1 -- 172.21.15.52:0/3258692692 wait complete. 2024-05-11T01:00:45.199 INFO:tasks.ceph:check_enable_crimson: False 2024-05-11T01:00:45.200 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-11T01:00:45.200 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:45.200 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-11T01:00:45.219 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:45.220+0000 7f741e410040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T01:00:45.225 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:45.224+0000 7fe21c11c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T01:00:45.226 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:45.226 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-11T01:00:45.292 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:45.292 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-11T01:00:45.357 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:00:45.358 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-11T01:00:45.388 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:45.388+0000 7f741e410040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:00:45.401 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:45.400+0000 7fe21c11c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:00:45.424 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:45.425 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-11T01:00:45.442 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:45.442+0000 7f741e410040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T01:00:45.450 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:45.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-11T01:00:45.457 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:45.457+0000 7fe21c11c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T01:00:45.515 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:45.515 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-11T01:00:45.581 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:00:45.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-11T01:00:45.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new ffc603c0-0cad-4c06-8325-aafc832216fa 0 2024-05-11T01:00:45.707 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:45.707+0000 7f741e410040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T01:00:45.713 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:45.713+0000 7fe21c11c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T01:00:45.755 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:45.755+0000 7f741e410040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T01:00:45.761 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:45.761+0000 7fe21c11c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T01:00:45.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.785+0000 7fc4c2a2a640 1 Processor -- start 2024-05-11T01:00:45.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.785+0000 7fc4c2a2a640 1 -- start start 2024-05-11T01:00:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4c2a2a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc4bc15d100 0x7fc4bc0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4c2a2a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4bc15d4e0 0x7fc4bc0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4c2a2a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4bc0ceef0 con 0x7fc4bc0b4a50 2024-05-11T01:00:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4c2a2a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc4bc0ce650 con 0x7fc4bc15d100 2024-05-11T01:00:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4c2a2a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc4bc0ce930 con 0x7fc4bc15d4e0 2024-05-11T01:00:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4bb7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4bc15d4e0 0x7fc4bc0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4c0fc7640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc4bc0b4a50 0x7fc4bc0cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52490/0 (socket says 172.21.15.119:52490) 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4bb7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4bc15d4e0 0x7fc4bc0cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36376/0 (socket says 172.21.15.119:36376) 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4c0fc7640 1 -- 172.21.15.119:0/3352732566 learned_addr learned my addr 172.21.15.119:0/3352732566 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4bbfff640 1 --2- 172.21.15.119:0/3352732566 >> v2:172.21.15.52:3300/0 conn(0x7fc4bc15d100 0x7fc4bc0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4bb7fe640 1 -- 172.21.15.119:0/3352732566 >> v2:172.21.15.52:3300/0 conn(0x7fc4bc15d100 msgr2=0x7fc4bc0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4bb7fe640 1 --2- 172.21.15.119:0/3352732566 >> v2:172.21.15.52:3300/0 conn(0x7fc4bc15d100 0x7fc4bc0cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4bb7fe640 1 -- 172.21.15.119:0/3352732566 >> v1:172.21.15.119:6789/0 conn(0x7fc4bc0b4a50 legacy=0x7fc4bc0cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4bb7fe640 1 -- 172.21.15.119:0/3352732566 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4bc0cec10 con 0x7fc4bc15d4e0 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.786+0000 7fc4baffd640 1 -- 172.21.15.119:0/3352732566 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2245092575 0 0) 0x7fc4bc0ceef0 con 0x7fc4bc0b4a50 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.787+0000 7fc4bb7fe640 1 --2- 172.21.15.119:0/3352732566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc15d4e0 0x7fc4bc0cdfd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0b5b70 tx=0x7fc4a402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d423231dcd23eac3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.787+0000 7fc4baffd640 1 -- 172.21.15.119:0/3352732566 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4037070 con 0x7fc4bc15d4e0 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.787+0000 7fc4baffd640 1 -- 172.21.15.119:0/3352732566 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a4032440 con 0x7fc4bc15d4e0 2024-05-11T01:00:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.787+0000 7fc4baffd640 1 -- 172.21.15.119:0/3352732566 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4032810 con 0x7fc4bc15d4e0 2024-05-11T01:00:45.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.788+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3352732566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc15d4e0 msgr2=0x7fc4bc0cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.788+0000 7fc4c2a2a640 1 --2- 172.21.15.119:0/3352732566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc15d4e0 0x7fc4bc0cdfd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0b5b70 tx=0x7fc4a402a1a0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.788+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3352732566 shutdown_connections 2024-05-11T01:00:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.788+0000 7fc4c2a2a640 1 --2- 172.21.15.119:0/3352732566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc15d4e0 0x7fc4bc0cdfd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.788+0000 7fc4c2a2a640 1 --2- 172.21.15.119:0/3352732566 >> v2:172.21.15.52:3300/0 conn(0x7fc4bc15d100 0x7fc4bc0cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.788+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3352732566 >> 172.21.15.119:0/3352732566 conn(0x7fc4bc0d2f80 msgr2=0x7fc4bc0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.788+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3352732566 shutdown_connections 2024-05-11T01:00:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.788+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3352732566 wait complete. 2024-05-11T01:00:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.789+0000 7fc4c2a2a640 1 Processor -- start 2024-05-11T01:00:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4c2a2a640 1 -- start start 2024-05-11T01:00:45.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4c2a2a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc0b4a50 0x7fc4bc1f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4c2a2a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4bc15d100 0x7fc4bc0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4c2a2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4bc15d4e0 0x7fc4bc0b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4c2a2a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc0b43e0 con 0x7fc4bc15d4e0 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4c2a2a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc0d0d80 con 0x7fc4bc0b4a50 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4c2a2a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc4bc0b4560 con 0x7fc4bc15d100 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4bbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc0b4a50 0x7fc4bc1f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4bbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc0b4a50 0x7fc4bc1f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36392/0 (socket says 172.21.15.119:36392) 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4bb7fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4bc15d100 0x7fc4bc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4bbfff640 1 -- 172.21.15.119:0/3171775914 learned_addr learned my addr 172.21.15.119:0/3171775914 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.790+0000 7fc4c0fc7640 1 --2- 172.21.15.119:0/3171775914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4bc15d4e0 0x7fc4bc0b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4bb7fe640 1 -- 172.21.15.119:0/3171775914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc0b4a50 msgr2=0x7fc4bc1f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4bb7fe640 1 --2- 172.21.15.119:0/3171775914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc0b4a50 0x7fc4bc1f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4bb7fe640 1 -- 172.21.15.119:0/3171775914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4bc15d4e0 msgr2=0x7fc4bc0b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4bb7fe640 1 --2- 172.21.15.119:0/3171775914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4bc15d4e0 0x7fc4bc0b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4bb7fe640 1 -- 172.21.15.119:0/3171775914 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4bc1f88b0 con 0x7fc4bc15d100 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4bb7fe640 1 --2- 172.21.15.119:0/3171775914 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4bc15d100 0x7fc4bc0b5b70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4031000 tx=0x7fc4a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4b8ff9640 1 -- 172.21.15.119:0/3171775914 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4041030 con 0x7fc4bc15d100 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4b8ff9640 1 -- 172.21.15.119:0/3171775914 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a4030500 con 0x7fc4bc15d100 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3171775914 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4bc1f8b90 con 0x7fc4bc15d100 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4b8ff9640 1 -- 172.21.15.119:0/3171775914 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4037040 con 0x7fc4bc15d100 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3171775914 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4bc1f9150 con 0x7fc4bc15d100 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4b8ff9640 1 -- 172.21.15.119:0/3171775914 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc4a4044070 con 0x7fc4bc15d100 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.791+0000 7fc4b8ff9640 1 -- 172.21.15.119:0/3171775914 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fc4a403dd40 con 0x7fc4bc15d100 2024-05-11T01:00:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.792+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3171775914 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4bc069c00 con 0x7fc4bc15d100 2024-05-11T01:00:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.794+0000 7fc4b8ff9640 1 -- 172.21.15.119:0/3171775914 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc4a403d420 con 0x7fc4bc15d100 2024-05-11T01:00:45.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.867+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3171775914 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ffc603c0-0cad-4c06-8325-aafc832216fa", "id": 0} v 0) -- 0x7fc4bc0d3bf0 con 0x7fc4bc15d100 2024-05-11T01:00:45.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.870+0000 7fc4b8ff9640 1 -- 172.21.15.119:0/3171775914 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ffc603c0-0cad-4c06-8325-aafc832216fa", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fc4a40308d0 con 0x7fc4bc15d100 2024-05-11T01:00:45.870 INFO:teuthology.orchestra.run.smithi119.stdout:0 2024-05-11T01:00:45.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.871+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3171775914 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4bc15d100 msgr2=0x7fc4bc0b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:45.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.871+0000 7fc4c2a2a640 1 --2- 172.21.15.119:0/3171775914 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4bc15d100 0x7fc4bc0b5b70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4031000 tx=0x7fc4a40046c0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.871+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3171775914 shutdown_connections 2024-05-11T01:00:45.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.871+0000 7fc4c2a2a640 1 --2- 172.21.15.119:0/3171775914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4bc15d4e0 0x7fc4bc0b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.871+0000 7fc4c2a2a640 1 --2- 172.21.15.119:0/3171775914 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4bc15d100 0x7fc4bc0b5b70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.871+0000 7fc4c2a2a640 1 --2- 172.21.15.119:0/3171775914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4bc0b4a50 0x7fc4bc1f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:45.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.871+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3171775914 >> 172.21.15.119:0/3171775914 conn(0x7fc4bc0d2f80 msgr2=0x7fc4bc0703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:45.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.871+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3171775914 shutdown_connections 2024-05-11T01:00:45.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:45.871+0000 7fc4c2a2a640 1 -- 172.21.15.119:0/3171775914 wait complete. 2024-05-11T01:00:45.881 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new ce9d9ccb-7863-4a96-82a3-011cfde3e799 1 2024-05-11T01:00:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.021+0000 7f9581664640 1 Processor -- start 2024-05-11T01:00:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.021+0000 7f9581664640 1 -- start start 2024-05-11T01:00:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f9581664640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c15f2f0 0x7f957c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f9581664640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f957c0d2d60 0x7f957c0d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f9581664640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f957c0d3a90 con 0x7f957c0d4e40 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f9581664640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f957c0d3d70 con 0x7f957c0d2d60 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f9581664640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f957c0d4050 con 0x7f957c15f2f0 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f957c0d2d60 0x7f957c0d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957affd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c15f2f0 0x7f957c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957affd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c15f2f0 0x7f957c0cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:56394/0 (socket says 172.21.15.119:56394) 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f957c0d2d60 0x7f957c0d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36398/0 (socket says 172.21.15.119:36398) 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957affd640 1 -- 172.21.15.119:0/3562223897 learned_addr learned my addr 172.21.15.119:0/3562223897 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957affd640 1 -- 172.21.15.119:0/3562223897 >> v2:172.21.15.119:3300/0 conn(0x7f957c0d2d60 msgr2=0x7f957c0d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957affd640 1 --2- 172.21.15.119:0/3562223897 >> v2:172.21.15.119:3300/0 conn(0x7f957c0d2d60 0x7f957c0d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957affd640 1 -- 172.21.15.119:0/3562223897 >> v1:172.21.15.52:6789/0 conn(0x7f957c0d4e40 legacy=0x7f957c0d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957affd640 1 -- 172.21.15.119:0/3562223897 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c0b4e60 con 0x7f957c15f2f0 2024-05-11T01:00:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.022+0000 7f957b7fe640 1 --2- 172.21.15.119:0/3562223897 >> v2:172.21.15.119:3300/0 conn(0x7f957c0d2d60 0x7f957c0d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:46.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.023+0000 7f957affd640 1 --2- 172.21.15.119:0/3562223897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c15f2f0 0x7f957c0cfca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f956c009870 tx=0x7f956c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eef8706a3560ed2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.023+0000 7f9579ffb640 1 -- 172.21.15.119:0/3562223897 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c037070 con 0x7f957c15f2f0 2024-05-11T01:00:46.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.023+0000 7f9579ffb640 1 -- 172.21.15.119:0/3562223897 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c032440 con 0x7f957c15f2f0 2024-05-11T01:00:46.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.023+0000 7f9579ffb640 1 -- 172.21.15.119:0/3562223897 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c032810 con 0x7f957c15f2f0 2024-05-11T01:00:46.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.024+0000 7f9581664640 1 -- 172.21.15.119:0/3562223897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c15f2f0 msgr2=0x7f957c0cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.024+0000 7f9581664640 1 --2- 172.21.15.119:0/3562223897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c15f2f0 0x7f957c0cfca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f956c009870 tx=0x7f956c02a270 comp rx=0 tx=0).stop 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.024+0000 7f9581664640 1 -- 172.21.15.119:0/3562223897 shutdown_connections 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.024+0000 7f9581664640 1 --2- 172.21.15.119:0/3562223897 >> v2:172.21.15.119:3300/0 conn(0x7f957c0d2d60 0x7f957c0d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.024+0000 7f9581664640 1 --2- 172.21.15.119:0/3562223897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c15f2f0 0x7f957c0cfca0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.024+0000 7f9581664640 1 -- 172.21.15.119:0/3562223897 >> 172.21.15.119:0/3562223897 conn(0x7f957c08ebb0 msgr2=0x7f957c09e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.024+0000 7f9581664640 1 -- 172.21.15.119:0/3562223897 shutdown_connections 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.024+0000 7f9581664640 1 -- 172.21.15.119:0/3562223897 wait complete. 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.025+0000 7f9581664640 1 Processor -- start 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.025+0000 7f9581664640 1 -- start start 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f9581664640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c0d2d60 0x7f957c0a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f9581664640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f957c0d4e40 0x7f957c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f9581664640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c15f2f0 0x7f957c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f9581664640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f957c0d01e0 con 0x7f957c0d4e40 2024-05-11T01:00:46.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f9581664640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f957c0d3710 con 0x7f957c15f2f0 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f9581664640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f957c0d0360 con 0x7f957c0d2d60 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957affd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c0d2d60 0x7f957c0a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c15f2f0 0x7f957c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c15f2f0 0x7f957c0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36408/0 (socket says 172.21.15.119:36408) 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957affd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c0d2d60 0x7f957c0a27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:56396/0 (socket says 172.21.15.119:56396) 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957b7fe640 1 -- 172.21.15.119:0/2554561320 learned_addr learned my addr 172.21.15.119:0/2554561320 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957a7fc640 1 --2- 172.21.15.119:0/2554561320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f957c0d4e40 0x7f957c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957affd640 1 -- 172.21.15.119:0/2554561320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c15f2f0 msgr2=0x7f957c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957affd640 1 --2- 172.21.15.119:0/2554561320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c15f2f0 0x7f957c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957affd640 1 -- 172.21.15.119:0/2554561320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f957c0d4e40 msgr2=0x7f957c15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957affd640 1 --2- 172.21.15.119:0/2554561320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f957c0d4e40 0x7f957c15a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957affd640 1 -- 172.21.15.119:0/2554561320 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c0ced40 con 0x7f957c0d2d60 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957b7fe640 1 --2- 172.21.15.119:0/2554561320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c15f2f0 0x7f957c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:46.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f957affd640 1 --2- 172.21.15.119:0/2554561320 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c0d2d60 0x7f957c0a27e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f956c031000 tx=0x7f956c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f95737fe640 1 -- 172.21.15.119:0/2554561320 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c041020 con 0x7f957c0d2d60 2024-05-11T01:00:46.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f95737fe640 1 -- 172.21.15.119:0/2554561320 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c030660 con 0x7f957c0d2d60 2024-05-11T01:00:46.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f9581664640 1 -- 172.21.15.119:0/2554561320 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c0cf020 con 0x7f957c0d2d60 2024-05-11T01:00:46.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f95737fe640 1 -- 172.21.15.119:0/2554561320 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c037040 con 0x7f957c0d2d60 2024-05-11T01:00:46.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.026+0000 7f9581664640 1 -- 172.21.15.119:0/2554561320 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c160590 con 0x7f957c0d2d60 2024-05-11T01:00:46.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.027+0000 7f95737fe640 1 -- 172.21.15.119:0/2554561320 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f956c044020 con 0x7f957c0d2d60 2024-05-11T01:00:46.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.027+0000 7f95737fe640 1 -- 172.21.15.119:0/2554561320 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f956c03dbe0 con 0x7f957c0d2d60 2024-05-11T01:00:46.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.027+0000 7f9581664640 1 -- 172.21.15.119:0/2554561320 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957c0695d0 con 0x7f957c0d2d60 2024-05-11T01:00:46.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.029+0000 7f95737fe640 1 -- 172.21.15.119:0/2554561320 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f956c04b030 con 0x7f957c0d2d60 2024-05-11T01:00:46.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.103+0000 7f9581664640 1 -- 172.21.15.119:0/2554561320 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ce9d9ccb-7863-4a96-82a3-011cfde3e799", "id": 1} v 0) -- 0x7f957c0d6080 con 0x7f957c0d2d60 2024-05-11T01:00:46.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.106+0000 7f95737fe640 1 -- 172.21.15.119:0/2554561320 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce9d9ccb-7863-4a96-82a3-011cfde3e799", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f956c03d420 con 0x7f957c0d2d60 2024-05-11T01:00:46.106 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-05-11T01:00:46.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.107+0000 7f9581664640 1 -- 172.21.15.119:0/2554561320 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c0d2d60 msgr2=0x7f957c0a27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.107+0000 7f9581664640 1 --2- 172.21.15.119:0/2554561320 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c0d2d60 0x7f957c0a27e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f956c031000 tx=0x7f956c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.107+0000 7f9581664640 1 -- 172.21.15.119:0/2554561320 shutdown_connections 2024-05-11T01:00:46.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.107+0000 7f9581664640 1 --2- 172.21.15.119:0/2554561320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c15f2f0 0x7f957c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.107+0000 7f9581664640 1 --2- 172.21.15.119:0/2554561320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f957c0d4e40 0x7f957c15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.107+0000 7f9581664640 1 --2- 172.21.15.119:0/2554561320 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f957c0d2d60 0x7f957c0a27e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.107+0000 7f9581664640 1 -- 172.21.15.119:0/2554561320 >> 172.21.15.119:0/2554561320 conn(0x7f957c08ebb0 msgr2=0x7f957c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.107+0000 7f9581664640 1 -- 172.21.15.119:0/2554561320 shutdown_connections 2024-05-11T01:00:46.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.107+0000 7f9581664640 1 -- 172.21.15.119:0/2554561320 wait complete. 2024-05-11T01:00:46.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 0f664c50-d358-46c6-b026-ef3106b46648 2 2024-05-11T01:00:46.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.218+0000 7f7cc9d6a640 1 Processor -- start 2024-05-11T01:00:46.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.218+0000 7f7cc9d6a640 1 -- start start 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc9d6a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7cc415f430 0x7f7cc40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc9d6a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7cc40b5420 0x7f7cc40d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc9d6a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc40d1eb0 0x7f7cc40d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc9d6a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc40d2a80 con 0x7f7cc40b5420 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc9d6a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc40cfc60 con 0x7f7cc415f430 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc9d6a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc4163120 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7cc415f430 0x7f7cc40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc3fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc40d1eb0 0x7f7cc40d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7cc415f430 0x7f7cc40b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36414/0 (socket says 172.21.15.119:36414) 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc37fe640 1 -- 172.21.15.119:0/3514440249 learned_addr learned my addr 172.21.15.119:0/3514440249 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc3fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc40d1eb0 0x7f7cc40d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:56406/0 (socket says 172.21.15.119:56406) 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc2ffd640 1 --2- 172.21.15.119:0/3514440249 >> v2:172.21.15.52:3300/0 conn(0x7f7cc40b5420 0x7f7cc40d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc37fe640 1 -- 172.21.15.119:0/3514440249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc40d1eb0 msgr2=0x7f7cc40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc37fe640 1 --2- 172.21.15.119:0/3514440249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc40d1eb0 0x7f7cc40d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc37fe640 1 -- 172.21.15.119:0/3514440249 >> v2:172.21.15.52:3300/0 conn(0x7f7cc40b5420 msgr2=0x7f7cc40d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc37fe640 1 --2- 172.21.15.119:0/3514440249 >> v2:172.21.15.52:3300/0 conn(0x7f7cc40b5420 0x7f7cc40d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc37fe640 1 -- 172.21.15.119:0/3514440249 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc40d24c0 con 0x7f7cc415f430 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.219+0000 7f7cc2ffd640 1 --2- 172.21.15.119:0/3514440249 >> v2:172.21.15.52:3300/0 conn(0x7f7cc40b5420 0x7f7cc40d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc37fe640 1 --2- 172.21.15.119:0/3514440249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc415f430 0x7f7cc40b4ee0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40d6290 tx=0x7f7cb402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d3b9d3934d54d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc27fc640 1 -- 172.21.15.119:0/3514440249 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4037070 con 0x7f7cc415f430 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc27fc640 1 -- 172.21.15.119:0/3514440249 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb4032440 con 0x7f7cc415f430 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc27fc640 1 -- 172.21.15.119:0/3514440249 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4032810 con 0x7f7cc415f430 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3514440249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc415f430 msgr2=0x7f7cc40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc9d6a640 1 --2- 172.21.15.119:0/3514440249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc415f430 0x7f7cc40b4ee0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40d6290 tx=0x7f7cb402a1e0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3514440249 shutdown_connections 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc9d6a640 1 --2- 172.21.15.119:0/3514440249 >> v2:172.21.15.52:3300/0 conn(0x7f7cc40b5420 0x7f7cc40d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc9d6a640 1 --2- 172.21.15.119:0/3514440249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc40d1eb0 0x7f7cc40d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc9d6a640 1 --2- 172.21.15.119:0/3514440249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc415f430 0x7f7cc40b4ee0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3514440249 >> 172.21.15.119:0/3514440249 conn(0x7f7cc408ebb0 msgr2=0x7f7cc409e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3514440249 shutdown_connections 2024-05-11T01:00:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.220+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3514440249 wait complete. 2024-05-11T01:00:46.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.222+0000 7f7cc9d6a640 1 Processor -- start 2024-05-11T01:00:46.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.222+0000 7f7cc9d6a640 1 -- start start 2024-05-11T01:00:46.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.222+0000 7f7cc9d6a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7cc40b5420 0x7f7cc415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.222+0000 7f7cc9d6a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc40d1eb0 0x7f7cc4161f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc9d6a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc415f430 0x7f7cc40c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc9d6a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc40d01e0 con 0x7f7cc40b5420 2024-05-11T01:00:46.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc9d6a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc40cff00 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc9d6a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc40d0c40 con 0x7f7cc415f430 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc40d1eb0 0x7f7cc4161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc40d1eb0 0x7f7cc4161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36426/0 (socket says 172.21.15.119:36426) 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc2ffd640 1 -- 172.21.15.119:0/3465123647 learned_addr learned my addr 172.21.15.119:0/3465123647 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc3fff640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc415f430 0x7f7cc40c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc37fe640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7cc40b5420 0x7f7cc415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc2ffd640 1 -- 172.21.15.119:0/3465123647 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc415f430 msgr2=0x7f7cc40c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc2ffd640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc415f430 0x7f7cc40c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc2ffd640 1 -- 172.21.15.119:0/3465123647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7cc40b5420 msgr2=0x7f7cc415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc2ffd640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7cc40b5420 0x7f7cc415d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc2ffd640 1 -- 172.21.15.119:0/3465123647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc40c8cc0 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc3fff640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc415f430 0x7f7cc40c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc2ffd640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc40d1eb0 0x7f7cc4161f70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800e7e0 tx=0x7f7cb800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cabfff640 1 -- 172.21.15.119:0/3465123647 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800f040 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cabfff640 1 -- 172.21.15.119:0/3465123647 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb8004030 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3465123647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc40c8fa0 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cabfff640 1 -- 172.21.15.119:0/3465123647 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8004400 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3465123647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc40c9840 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cabfff640 1 -- 172.21.15.119:0/3465123647 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7cb800ca60 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.223+0000 7f7cabfff640 1 -- 172.21.15.119:0/3465123647 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f7cb8013970 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.224+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3465123647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc40c9170 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.226+0000 7f7cabfff640 1 -- 172.21.15.119:0/3465123647 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7cb8017020 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.298+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3465123647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0f664c50-d358-46c6-b026-ef3106b46648", "id": 2} v 0) -- 0x7f7cc40695d0 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.302+0000 7f7cabfff640 1 -- 172.21.15.119:0/3465123647 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0f664c50-d358-46c6-b026-ef3106b46648", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f7cb8013c20 con 0x7f7cc40d1eb0 2024-05-11T01:00:46.302 INFO:teuthology.orchestra.run.smithi119.stdout:2 2024-05-11T01:00:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.302+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3465123647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc40d1eb0 msgr2=0x7f7cc4161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.302+0000 7f7cc9d6a640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc40d1eb0 0x7f7cc4161f70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800e7e0 tx=0x7f7cb800eca0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.303+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3465123647 shutdown_connections 2024-05-11T01:00:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.303+0000 7f7cc9d6a640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7cc415f430 0x7f7cc40c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.303+0000 7f7cc9d6a640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cc40d1eb0 0x7f7cc4161f70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.303+0000 7f7cc9d6a640 1 --2- 172.21.15.119:0/3465123647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7cc40b5420 0x7f7cc415d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.303+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3465123647 >> 172.21.15.119:0/3465123647 conn(0x7f7cc408ebb0 msgr2=0x7f7cc415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.303+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3465123647 shutdown_connections 2024-05-11T01:00:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.303+0000 7f7cc9d6a640 1 -- 172.21.15.119:0/3465123647 wait complete. 2024-05-11T01:00:46.313 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 26330d08-2e2a-4cdf-bccd-100d145b1e43 3 2024-05-11T01:00:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.413+0000 7fdce0809640 1 Processor -- start 2024-05-11T01:00:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.413+0000 7fdce0809640 1 -- start start 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.414+0000 7fdce0809640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdcdc159af0 0x7fdcdc15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.414+0000 7fdce0809640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdcdc0b3c20 0x7fdcdc0b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.414+0000 7fdce0809640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcdc159ed0 con 0x7fdcdc0b0700 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.414+0000 7fdce0809640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdcdc0af810 con 0x7fdcdc159af0 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.414+0000 7fdce0809640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fdcdc06d000 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.414+0000 7fdcdad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdcdc0b3c20 0x7fdcdc0b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.414+0000 7fdcdad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdcdc0b3c20 0x7fdcdc0b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36434/0 (socket says 172.21.15.119:36434) 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.414+0000 7fdcdad9d640 1 -- 172.21.15.119:0/2481145819 learned_addr learned my addr 172.21.15.119:0/2481145819 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.414+0000 7fdcda59c640 1 --2- 172.21.15.119:0/2481145819 >> v2:172.21.15.52:3300/0 conn(0x7fdcdc159af0 0x7fdcdc15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.415+0000 7fdcdad9d640 1 -- 172.21.15.119:0/2481145819 >> v2:172.21.15.52:3300/0 conn(0x7fdcdc159af0 msgr2=0x7fdcdc15fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.415+0000 7fdcdad9d640 1 --2- 172.21.15.119:0/2481145819 >> v2:172.21.15.52:3300/0 conn(0x7fdcdc159af0 0x7fdcdc15fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.415+0000 7fdcdad9d640 1 -- 172.21.15.119:0/2481145819 >> v1:172.21.15.52:6789/0 conn(0x7fdcdc0b0700 legacy=0x7fdcdc163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.415+0000 7fdcdad9d640 1 -- 172.21.15.119:0/2481145819 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc0aec00 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.416+0000 7fdcdad9d640 1 --2- 172.21.15.119:0/2481145819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 0x7fdcdc0b0b80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0b01f0 tx=0x7fdcd402a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fbfaf0aa4037f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.416+0000 7fdcd959a640 1 -- 172.21.15.119:0/2481145819 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4036070 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.416+0000 7fdcd959a640 1 -- 172.21.15.119:0/2481145819 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd4031600 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.416+0000 7fdcd959a640 1 -- 172.21.15.119:0/2481145819 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40319d0 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.416+0000 7fdce0809640 1 -- 172.21.15.119:0/2481145819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 msgr2=0x7fdcdc0b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.416+0000 7fdce0809640 1 --2- 172.21.15.119:0/2481145819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 0x7fdcdc0b0b80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0b01f0 tx=0x7fdcd402a6a0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.417+0000 7fdce0809640 1 -- 172.21.15.119:0/2481145819 shutdown_connections 2024-05-11T01:00:46.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.417+0000 7fdce0809640 1 --2- 172.21.15.119:0/2481145819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 0x7fdcdc0b0b80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.417+0000 7fdce0809640 1 --2- 172.21.15.119:0/2481145819 >> v2:172.21.15.52:3300/0 conn(0x7fdcdc159af0 0x7fdcdc15fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.417+0000 7fdce0809640 1 -- 172.21.15.119:0/2481145819 >> 172.21.15.119:0/2481145819 conn(0x7fdcdc076090 msgr2=0x7fdcdc06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.417+0000 7fdce0809640 1 -- 172.21.15.119:0/2481145819 shutdown_connections 2024-05-11T01:00:46.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.417+0000 7fdce0809640 1 -- 172.21.15.119:0/2481145819 wait complete. 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.418+0000 7fdce0809640 1 Processor -- start 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.418+0000 7fdce0809640 1 -- start start 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdce0809640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcdc0b0700 0x7fdcdc1f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdce0809640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 0x7fdcdc15d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdce0809640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcdc159af0 0x7fdcdc15db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdce0809640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0af4a0 con 0x7fdcdc159af0 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdce0809640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0af1c0 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdce0809640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc06cb90 con 0x7fdcdc0b0700 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcdc0b0700 0x7fdcdc1f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd9d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 0x7fdcdc15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcdc0b0700 0x7fdcdc1f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:56424/0 (socket says 172.21.15.119:56424) 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd9d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 0x7fdcdc15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36448/0 (socket says 172.21.15.119:36448) 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd9d9b640 1 -- 172.21.15.119:0/3006219595 learned_addr learned my addr 172.21.15.119:0/3006219595 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcdad9d640 1 --2- 172.21.15.119:0/3006219595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcdc159af0 0x7fdcdc15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd9d9b640 1 -- 172.21.15.119:0/3006219595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcdc0b0700 msgr2=0x7fdcdc1f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd9d9b640 1 --2- 172.21.15.119:0/3006219595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcdc0b0700 0x7fdcdc1f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd9d9b640 1 -- 172.21.15.119:0/3006219595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcdc159af0 msgr2=0x7fdcdc15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd9d9b640 1 --2- 172.21.15.119:0/3006219595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcdc159af0 0x7fdcdc15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd9d9b640 1 -- 172.21.15.119:0/3006219595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc14bb60 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd9d9b640 1 --2- 172.21.15.119:0/3006219595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 0x7fdcdc15d630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c3a0 tx=0x7fdcc400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.419+0000 7fdcd2ffd640 1 -- 172.21.15.119:0/3006219595 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4017070 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.420+0000 7fdcd2ffd640 1 -- 172.21.15.119:0/3006219595 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc40052d0 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.420+0000 7fdce0809640 1 -- 172.21.15.119:0/3006219595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc1f4be0 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.420+0000 7fdcd2ffd640 1 -- 172.21.15.119:0/3006219595 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4003830 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.420+0000 7fdce0809640 1 -- 172.21.15.119:0/3006219595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc15ca50 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.420+0000 7fdcd2ffd640 1 -- 172.21.15.119:0/3006219595 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdcc401a070 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.420+0000 7fdcd2ffd640 1 -- 172.21.15.119:0/3006219595 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fdcc4012420 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.420+0000 7fdce0809640 1 -- 172.21.15.119:0/3006219595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc069c00 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.424+0000 7fdcd2ffd640 1 -- 172.21.15.119:0/3006219595 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdcc4021070 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.497+0000 7fdce0809640 1 -- 172.21.15.119:0/3006219595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "26330d08-2e2a-4cdf-bccd-100d145b1e43", "id": 3} v 0) -- 0x7fdcdc14c510 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.501+0000 7fdcd2ffd640 1 -- 172.21.15.119:0/3006219595 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "26330d08-2e2a-4cdf-bccd-100d145b1e43", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fdcdc1f4be0 con 0x7fdcdc0b3c20 2024-05-11T01:00:46.501 INFO:teuthology.orchestra.run.smithi119.stdout:3 2024-05-11T01:00:46.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.501+0000 7fdce0809640 1 -- 172.21.15.119:0/3006219595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 msgr2=0x7fdcdc15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.501+0000 7fdce0809640 1 --2- 172.21.15.119:0/3006219595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 0x7fdcdc15d630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c3a0 tx=0x7fdcc400c860 comp rx=0 tx=0).stop 2024-05-11T01:00:46.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.501+0000 7fdce0809640 1 -- 172.21.15.119:0/3006219595 shutdown_connections 2024-05-11T01:00:46.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.502+0000 7fdce0809640 1 --2- 172.21.15.119:0/3006219595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcdc159af0 0x7fdcdc15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.502+0000 7fdce0809640 1 --2- 172.21.15.119:0/3006219595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcdc0b3c20 0x7fdcdc15d630 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.502+0000 7fdce0809640 1 --2- 172.21.15.119:0/3006219595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdcdc0b0700 0x7fdcdc1f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.502+0000 7fdce0809640 1 -- 172.21.15.119:0/3006219595 >> 172.21.15.119:0/3006219595 conn(0x7fdcdc076090 msgr2=0x7fdcdc163970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.502+0000 7fdce0809640 1 -- 172.21.15.119:0/3006219595 shutdown_connections 2024-05-11T01:00:46.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.502+0000 7fdce0809640 1 -- 172.21.15.119:0/3006219595 wait complete. 2024-05-11T01:00:46.512 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new fcabd274-e244-4a4c-a548-3436c80d0ec0 4 2024-05-11T01:00:46.513 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:46.512+0000 7fe21c11c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T01:00:46.555 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:46.555+0000 7f741e410040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T01:00:46.604 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:46.604+0000 7fe21c11c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T01:00:46.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.611+0000 7f897a8a0640 1 Processor -- start 2024-05-11T01:00:46.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.611+0000 7f897a8a0640 1 -- start start 2024-05-11T01:00:46.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f897a8a0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8974159af0 0x7f897415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f897a8a0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89740b0700 0x7f89740b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f897a8a0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8974159ed0 con 0x7f8974163030 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f897a8a0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f89740af810 con 0x7f8974159af0 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f897a8a0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f897406d000 con 0x7f89740b0700 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f896bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89740b0700 0x7f89740b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f8978e3d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8974163030 0x7f8974163460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52496/0 (socket says 172.21.15.119:52496) 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f8978e3d640 1 -- 172.21.15.119:0/2567846238 learned_addr learned my addr 172.21.15.119:0/2567846238 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f8973fff640 1 --2- 172.21.15.119:0/2567846238 >> v2:172.21.15.52:3300/0 conn(0x7f8974159af0 0x7f897415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f896bfff640 1 -- 172.21.15.119:0/2567846238 >> v2:172.21.15.52:3300/0 conn(0x7f8974159af0 msgr2=0x7f897415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f896bfff640 1 --2- 172.21.15.119:0/2567846238 >> v2:172.21.15.52:3300/0 conn(0x7f8974159af0 0x7f897415fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f896bfff640 1 -- 172.21.15.119:0/2567846238 >> v1:172.21.15.119:6789/0 conn(0x7f8974163030 legacy=0x7f8974163460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.612+0000 7f896bfff640 1 -- 172.21.15.119:0/2567846238 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89740aec00 con 0x7f89740b0700 2024-05-11T01:00:46.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.613+0000 7f896bfff640 1 --2- 172.21.15.119:0/2567846238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 0x7f89740b0b80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f89741640f0 tx=0x7f895c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2466a08f0e326 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.613+0000 7f89737fe640 1 -- 172.21.15.119:0/2567846238 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c037070 con 0x7f89740b0700 2024-05-11T01:00:46.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.613+0000 7f89737fe640 1 -- 172.21.15.119:0/2567846238 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c032440 con 0x7f89740b0700 2024-05-11T01:00:46.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.613+0000 7f89737fe640 1 -- 172.21.15.119:0/2567846238 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c032810 con 0x7f89740b0700 2024-05-11T01:00:46.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.614+0000 7f897a8a0640 1 -- 172.21.15.119:0/2567846238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 msgr2=0x7f89740b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.614+0000 7f897a8a0640 1 --2- 172.21.15.119:0/2567846238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 0x7f89740b0b80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f89741640f0 tx=0x7f895c02a1a0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.614+0000 7f897a8a0640 1 -- 172.21.15.119:0/2567846238 shutdown_connections 2024-05-11T01:00:46.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.614+0000 7f897a8a0640 1 --2- 172.21.15.119:0/2567846238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 0x7f89740b0b80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.614+0000 7f897a8a0640 1 --2- 172.21.15.119:0/2567846238 >> v2:172.21.15.52:3300/0 conn(0x7f8974159af0 0x7f897415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.614+0000 7f897a8a0640 1 -- 172.21.15.119:0/2567846238 >> 172.21.15.119:0/2567846238 conn(0x7f8974076090 msgr2=0x7f897406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.614+0000 7f897a8a0640 1 -- 172.21.15.119:0/2567846238 shutdown_connections 2024-05-11T01:00:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.614+0000 7f897a8a0640 1 -- 172.21.15.119:0/2567846238 wait complete. 2024-05-11T01:00:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.615+0000 7f897a8a0640 1 Processor -- start 2024-05-11T01:00:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.615+0000 7f897a8a0640 1 -- start start 2024-05-11T01:00:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f897a8a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 0x7f8974144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f897a8a0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8974159af0 0x7f89741547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f897a8a0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8974163030 0x7f897406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f897a8a0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89740af4a0 con 0x7f8974163030 2024-05-11T01:00:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f897a8a0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89740af1c0 con 0x7f89740b0700 2024-05-11T01:00:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f897a8a0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f897406cb90 con 0x7f8974159af0 2024-05-11T01:00:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f8973fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 0x7f8974144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f896bfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8974159af0 0x7f89741547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f8973fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 0x7f8974144fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36460/0 (socket says 172.21.15.119:36460) 2024-05-11T01:00:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f8973fff640 1 -- 172.21.15.119:0/3601789958 learned_addr learned my addr 172.21.15.119:0/3601789958 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f8978e3d640 1 --2- 172.21.15.119:0/3601789958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8974163030 0x7f897406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f896bfff640 1 -- 172.21.15.119:0/3601789958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 msgr2=0x7f8974144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f896bfff640 1 --2- 172.21.15.119:0/3601789958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 0x7f8974144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f896bfff640 1 -- 172.21.15.119:0/3601789958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8974163030 msgr2=0x7f897406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f896bfff640 1 --2- 172.21.15.119:0/3601789958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8974163030 0x7f897406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f896bfff640 1 -- 172.21.15.119:0/3601789958 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8974145520 con 0x7f8974159af0 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.616+0000 7f8973fff640 1 --2- 172.21.15.119:0/3601789958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 0x7f8974144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.617+0000 7f896bfff640 1 --2- 172.21.15.119:0/3601789958 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8974159af0 0x7f89741547b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f895c031000 tx=0x7f895c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.617+0000 7f89717fa640 1 -- 172.21.15.119:0/3601789958 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c041030 con 0x7f8974159af0 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.617+0000 7f89717fa640 1 -- 172.21.15.119:0/3601789958 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c030500 con 0x7f8974159af0 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.617+0000 7f897a8a0640 1 -- 172.21.15.119:0/3601789958 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89741539f0 con 0x7f8974159af0 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.617+0000 7f89717fa640 1 -- 172.21.15.119:0/3601789958 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c037040 con 0x7f8974159af0 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.617+0000 7f897a8a0640 1 -- 172.21.15.119:0/3601789958 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8974153fb0 con 0x7f8974159af0 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.617+0000 7f89717fa640 1 -- 172.21.15.119:0/3601789958 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f895c044070 con 0x7f8974159af0 2024-05-11T01:00:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.617+0000 7f89717fa640 1 -- 172.21.15.119:0/3601789958 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f895c03db60 con 0x7f8974159af0 2024-05-11T01:00:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.618+0000 7f897a8a0640 1 -- 172.21.15.119:0/3601789958 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8974069c00 con 0x7f8974159af0 2024-05-11T01:00:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.620+0000 7f89717fa640 1 -- 172.21.15.119:0/3601789958 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f895c03de00 con 0x7f8974159af0 2024-05-11T01:00:46.648 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:46.648+0000 7f741e410040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T01:00:46.649 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:46.650+0000 7fe21c11c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T01:00:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.692+0000 7f897a8a0640 1 -- 172.21.15.119:0/3601789958 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fcabd274-e244-4a4c-a548-3436c80d0ec0", "id": 4} v 0) -- 0x7f89740d09d0 con 0x7f8974159af0 2024-05-11T01:00:46.694 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:46.692+0000 7f741e410040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T01:00:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.695+0000 7f89717fa640 1 -- 172.21.15.119:0/3601789958 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fcabd274-e244-4a4c-a548-3436c80d0ec0", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f895c03d650 con 0x7f8974159af0 2024-05-11T01:00:46.695 INFO:teuthology.orchestra.run.smithi119.stdout:4 2024-05-11T01:00:46.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.696+0000 7f897a8a0640 1 -- 172.21.15.119:0/3601789958 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8974159af0 msgr2=0x7f89741547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.696+0000 7f897a8a0640 1 --2- 172.21.15.119:0/3601789958 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8974159af0 0x7f89741547b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f895c031000 tx=0x7f895c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.696+0000 7f897a8a0640 1 -- 172.21.15.119:0/3601789958 shutdown_connections 2024-05-11T01:00:46.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.696+0000 7f897a8a0640 1 --2- 172.21.15.119:0/3601789958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8974163030 0x7f897406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.696+0000 7f897a8a0640 1 --2- 172.21.15.119:0/3601789958 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8974159af0 0x7f89741547b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.696+0000 7f897a8a0640 1 --2- 172.21.15.119:0/3601789958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89740b0700 0x7f8974144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.696+0000 7f897a8a0640 1 -- 172.21.15.119:0/3601789958 >> 172.21.15.119:0/3601789958 conn(0x7f8974076090 msgr2=0x7f89741590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.696+0000 7f897a8a0640 1 -- 172.21.15.119:0/3601789958 shutdown_connections 2024-05-11T01:00:46.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.696+0000 7f897a8a0640 1 -- 172.21.15.119:0/3601789958 wait complete. 2024-05-11T01:00:46.707 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8 5 2024-05-11T01:00:46.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.806+0000 7fc9041da640 1 Processor -- start 2024-05-11T01:00:46.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.806+0000 7fc9041da640 1 -- start start 2024-05-11T01:00:46.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc9041da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc9041da640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8fc073b30 con 0x7fc8fc073250 2024-05-11T01:00:46.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc9041da640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8fc070d50 con 0x7fc8fc0d1270 2024-05-11T01:00:46.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc9041da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc8fc06f430 con 0x7fc8fc0736e0 2024-05-11T01:00:46.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc902777640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc901f76640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc8fc0d1270 0x7fc8fc0d1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52498/0 (socket says 172.21.15.119:52498) 2024-05-11T01:00:46.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc901f76640 1 -- 172.21.15.119:0/560343021 learned_addr learned my addr 172.21.15.119:0/560343021 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-05-11T01:00:46.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc902777640 1 -- 172.21.15.119:0/560343021 >> v1:172.21.15.119:6789/0 conn(0x7fc8fc0d1270 legacy=0x7fc8fc0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc902777640 1 -- 172.21.15.119:0/560343021 >> v1:172.21.15.52:6789/0 conn(0x7fc8fc073250 legacy=0x7fc8fc0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc902777640 1 -- 172.21.15.119:0/560343021 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc06f710 con 0x7fc8fc0736e0 2024-05-11T01:00:46.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.807+0000 7fc900f74640 1 -- 172.21.15.119:0/560343021 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 958251845 0 0) 0x7fc8fc070d50 con 0x7fc8fc0d1270 2024-05-11T01:00:46.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.808+0000 7fc902777640 1 --2- 172.21.15.119:0/560343021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0ccbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009dc0 tx=0x7fc8f402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33f5622b962b5bab server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.808+0000 7fc900f74640 1 -- 172.21.15.119:0/560343021 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4007740 con 0x7fc8fc0736e0 2024-05-11T01:00:46.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.808+0000 7fc900f74640 1 -- 172.21.15.119:0/560343021 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f40078e0 con 0x7fc8fc0736e0 2024-05-11T01:00:46.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.808+0000 7fc900f74640 1 -- 172.21.15.119:0/560343021 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4007cb0 con 0x7fc8fc0736e0 2024-05-11T01:00:46.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.809+0000 7fc9041da640 1 -- 172.21.15.119:0/560343021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 msgr2=0x7fc8fc0ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.809+0000 7fc9041da640 1 --2- 172.21.15.119:0/560343021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0ccbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009dc0 tx=0x7fc8f402a6d0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.809+0000 7fc9041da640 1 -- 172.21.15.119:0/560343021 shutdown_connections 2024-05-11T01:00:46.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.809+0000 7fc9041da640 1 --2- 172.21.15.119:0/560343021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0ccbb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.809+0000 7fc9041da640 1 -- 172.21.15.119:0/560343021 >> 172.21.15.119:0/560343021 conn(0x7fc8fc0d1e50 msgr2=0x7fc8fc0d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.809+0000 7fc9041da640 1 -- 172.21.15.119:0/560343021 shutdown_connections 2024-05-11T01:00:46.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.809+0000 7fc9041da640 1 -- 172.21.15.119:0/560343021 wait complete. 2024-05-11T01:00:46.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.810+0000 7fc9041da640 1 Processor -- start 2024-05-11T01:00:46.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.810+0000 7fc9041da640 1 -- start start 2024-05-11T01:00:46.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc9041da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8fc073250 0x7fc8fc1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc9041da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc9041da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8fc0d1270 0x7fc8fc0b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:46.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc9041da640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc15f5e0 con 0x7fc8fc0d1270 2024-05-11T01:00:46.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc9041da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc06fcd0 con 0x7fc8fc073250 2024-05-11T01:00:46.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc9041da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc8fc15f760 con 0x7fc8fc0736e0 2024-05-11T01:00:46.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901f76640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8fc073250 0x7fc8fc1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901775640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901f76640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8fc073250 0x7fc8fc1511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36470/0 (socket says 172.21.15.119:36470) 2024-05-11T01:00:46.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901f76640 1 -- 172.21.15.119:0/2982449646 learned_addr learned my addr 172.21.15.119:0/2982449646 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:46.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc902777640 1 --2- 172.21.15.119:0/2982449646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8fc0d1270 0x7fc8fc0b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:46.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901775640 1 -- 172.21.15.119:0/2982449646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8fc073250 msgr2=0x7fc8fc1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901775640 1 --2- 172.21.15.119:0/2982449646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8fc073250 0x7fc8fc1511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901775640 1 -- 172.21.15.119:0/2982449646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8fc0d1270 msgr2=0x7fc8fc0b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901775640 1 --2- 172.21.15.119:0/2982449646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8fc0d1270 0x7fc8fc0b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901775640 1 -- 172.21.15.119:0/2982449646 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc1619b0 con 0x7fc8fc0736e0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc901775640 1 --2- 172.21.15.119:0/2982449646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0b7660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00c340 tx=0x7fc8ec00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc8f27fc640 1 -- 172.21.15.119:0/2982449646 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec017070 con 0x7fc8fc0736e0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc8f27fc640 1 -- 172.21.15.119:0/2982449646 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ec0052d0 con 0x7fc8fc0736e0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc9041da640 1 -- 172.21.15.119:0/2982449646 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc161c90 con 0x7fc8fc0736e0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc8f27fc640 1 -- 172.21.15.119:0/2982449646 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec003810 con 0x7fc8fc0736e0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc9041da640 1 -- 172.21.15.119:0/2982449646 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc162530 con 0x7fc8fc0736e0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.811+0000 7fc8f27fc640 1 -- 172.21.15.119:0/2982449646 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc8ec01a070 con 0x7fc8fc0736e0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.812+0000 7fc8f27fc640 1 -- 172.21.15.119:0/2982449646 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fc8ec011ae0 con 0x7fc8fc0736e0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.812+0000 7fc9041da640 1 -- 172.21.15.119:0/2982449646 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8fc161e60 con 0x7fc8fc0736e0 2024-05-11T01:00:46.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.814+0000 7fc8f27fc640 1 -- 172.21.15.119:0/2982449646 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc8ec021020 con 0x7fc8fc0736e0 2024-05-11T01:00:46.824 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:46.824+0000 7fe21c11c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T01:00:46.869 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:46.869+0000 7f741e410040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T01:00:46.883 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:46.884+0000 7fe21c11c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T01:00:46.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.887+0000 7fc9041da640 1 -- 172.21.15.119:0/2982449646 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8", "id": 5} v 0) -- 0x7fc8fc069c00 con 0x7fc8fc0736e0 2024-05-11T01:00:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.890+0000 7fc8f27fc640 1 -- 172.21.15.119:0/2982449646 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fc8ec011da0 con 0x7fc8fc0736e0 2024-05-11T01:00:46.890 INFO:teuthology.orchestra.run.smithi119.stdout:5 2024-05-11T01:00:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.891+0000 7fc9041da640 1 -- 172.21.15.119:0/2982449646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 msgr2=0x7fc8fc0b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.891+0000 7fc9041da640 1 --2- 172.21.15.119:0/2982449646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0b7660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00c340 tx=0x7fc8ec00c800 comp rx=0 tx=0).stop 2024-05-11T01:00:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.891+0000 7fc9041da640 1 -- 172.21.15.119:0/2982449646 shutdown_connections 2024-05-11T01:00:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.891+0000 7fc9041da640 1 --2- 172.21.15.119:0/2982449646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8fc0d1270 0x7fc8fc0b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.891+0000 7fc9041da640 1 --2- 172.21.15.119:0/2982449646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc8fc0736e0 0x7fc8fc0b7660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.891+0000 7fc9041da640 1 --2- 172.21.15.119:0/2982449646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8fc073250 0x7fc8fc1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.891+0000 7fc9041da640 1 -- 172.21.15.119:0/2982449646 >> 172.21.15.119:0/2982449646 conn(0x7fc8fc0d1e50 msgr2=0x7fc8fc06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.891+0000 7fc9041da640 1 -- 172.21.15.119:0/2982449646 shutdown_connections 2024-05-11T01:00:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:46.891+0000 7fc9041da640 1 -- 172.21.15.119:0/2982449646 wait complete. 2024-05-11T01:00:46.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new d674a25d-afaa-4e54-802f-a352933abb02 6 2024-05-11T01:00:46.921 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:46.920+0000 7f741e410040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T01:00:46.939 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:46.939+0000 7fe21c11c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T01:00:46.976 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:46.976+0000 7f741e410040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T01:00:47.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.000+0000 7f87355b6640 1 Processor -- start 2024-05-11T01:00:47.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.000+0000 7f87355b6640 1 -- start start 2024-05-11T01:00:47.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.001+0000 7f87355b6640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300ceca0 0x7f8730169f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.001+0000 7f87355b6640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87300d1d20 con 0x7f87300d1030 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.001+0000 7f87355b6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87300d1fc0 con 0x7f87300cf940 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.001+0000 7f87355b6640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f87300cd800 con 0x7f87300ceca0 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.001+0000 7f872f7fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300ceca0 0x7f8730169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.001+0000 7f872effd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f87300cf940 0x7f87300cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52510/0 (socket says 172.21.15.119:52510) 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.001+0000 7f872effd640 1 -- 172.21.15.119:0/801147761 learned_addr learned my addr 172.21.15.119:0/801147761 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.002+0000 7f872f7fe640 1 -- 172.21.15.119:0/801147761 >> v1:172.21.15.119:6789/0 conn(0x7f87300cf940 legacy=0x7f87300cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.002+0000 7f872dffb640 1 -- 172.21.15.119:0/801147761 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2140231959 0 0) 0x7f87300d1fc0 con 0x7f87300cf940 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.002+0000 7f872f7fe640 1 -- 172.21.15.119:0/801147761 >> v1:172.21.15.52:6789/0 conn(0x7f87300d1030 legacy=0x7f87300d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.002+0000 7f872f7fe640 1 -- 172.21.15.119:0/801147761 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87300d2260 con 0x7f87300ceca0 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.002+0000 7f872f7fe640 1 --2- 172.21.15.119:0/801147761 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300ceca0 0x7f8730169f10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8728009dc0 tx=0x7f872802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333ae1c7708f1204 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.002+0000 7f872dffb640 1 -- 172.21.15.119:0/801147761 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728007740 con 0x7f87300ceca0 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.002+0000 7f872dffb640 1 -- 172.21.15.119:0/801147761 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87280078e0 con 0x7f87300ceca0 2024-05-11T01:00:47.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.002+0000 7f872dffb640 1 -- 172.21.15.119:0/801147761 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728007cb0 con 0x7f87300ceca0 2024-05-11T01:00:47.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.003+0000 7f87355b6640 1 -- 172.21.15.119:0/801147761 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300ceca0 msgr2=0x7f8730169f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.003+0000 7f87355b6640 1 --2- 172.21.15.119:0/801147761 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300ceca0 0x7f8730169f10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8728009dc0 tx=0x7f872802a6d0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.003+0000 7f87355b6640 1 -- 172.21.15.119:0/801147761 shutdown_connections 2024-05-11T01:00:47.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.003+0000 7f87355b6640 1 --2- 172.21.15.119:0/801147761 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300ceca0 0x7f8730169f10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.003+0000 7f87355b6640 1 -- 172.21.15.119:0/801147761 >> 172.21.15.119:0/801147761 conn(0x7f87300d2fa0 msgr2=0x7f87300d33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:47.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.003+0000 7f87355b6640 1 -- 172.21.15.119:0/801147761 shutdown_connections 2024-05-11T01:00:47.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.003+0000 7f87355b6640 1 -- 172.21.15.119:0/801147761 wait complete. 2024-05-11T01:00:47.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.004+0000 7f87355b6640 1 Processor -- start 2024-05-11T01:00:47.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.004+0000 7f87355b6640 1 -- start start 2024-05-11T01:00:47.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f87355b6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300ceca0 0x7f873015b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f87355b6640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300cf940 0x7f8730153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f87355b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87300d1030 0x7f87300c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f87355b6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f873016a5d0 con 0x7f87300ceca0 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f87355b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f873016a450 con 0x7f87300d1030 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f87355b6640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f873016a750 con 0x7f87300cf940 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f872f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87300d1030 0x7f87300c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f872f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87300d1030 0x7f87300c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36474/0 (socket says 172.21.15.119:36474) 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f872f7fe640 1 -- 172.21.15.119:0/690159018 learned_addr learned my addr 172.21.15.119:0/690159018 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f872e7fc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300cf940 0x7f8730153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.005+0000 7f872effd640 1 --2- 172.21.15.119:0/690159018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300ceca0 0x7f873015b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f872f7fe640 1 -- 172.21.15.119:0/690159018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300cf940 msgr2=0x7f8730153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f872f7fe640 1 --2- 172.21.15.119:0/690159018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300cf940 0x7f8730153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f872f7fe640 1 -- 172.21.15.119:0/690159018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300ceca0 msgr2=0x7f873015b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f872f7fe640 1 --2- 172.21.15.119:0/690159018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300ceca0 0x7f873015b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f872f7fe640 1 -- 172.21.15.119:0/690159018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8730160450 con 0x7f87300d1030 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f872e7fc640 1 --2- 172.21.15.119:0/690159018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300cf940 0x7f8730153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f872f7fe640 1 --2- 172.21.15.119:0/690159018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87300d1030 0x7f87300c8f20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f872802acc0 tx=0x7f8728004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f87277fe640 1 -- 172.21.15.119:0/690159018 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728040070 con 0x7f87300d1030 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f87277fe640 1 -- 172.21.15.119:0/690159018 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8728031bc0 con 0x7f87300d1030 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f87355b6640 1 -- 172.21.15.119:0/690159018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8730160730 con 0x7f87300d1030 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f87277fe640 1 -- 172.21.15.119:0/690159018 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728003990 con 0x7f87300d1030 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f87355b6640 1 -- 172.21.15.119:0/690159018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8730160cf0 con 0x7f87300d1030 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f87277fe640 1 -- 172.21.15.119:0/690159018 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8728039040 con 0x7f87300d1030 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.006+0000 7f87277fe640 1 -- 172.21.15.119:0/690159018 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f8728004800 con 0x7f87300d1030 2024-05-11T01:00:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.007+0000 7f87355b6640 1 -- 172.21.15.119:0/690159018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8730069c00 con 0x7f87300d1030 2024-05-11T01:00:47.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.009+0000 7f87277fe640 1 -- 172.21.15.119:0/690159018 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8728007840 con 0x7f87300d1030 2024-05-11T01:00:47.057 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:47.057+0000 7fe21c11c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:00:47.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.082+0000 7f87355b6640 1 -- 172.21.15.119:0/690159018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d674a25d-afaa-4e54-802f-a352933abb02", "id": 6} v 0) -- 0x7f8730071cd0 con 0x7f87300d1030 2024-05-11T01:00:47.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.085+0000 7f87277fe640 1 -- 172.21.15.119:0/690159018 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d674a25d-afaa-4e54-802f-a352933abb02", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f8728004030 con 0x7f87300d1030 2024-05-11T01:00:47.085 INFO:teuthology.orchestra.run.smithi119.stdout:6 2024-05-11T01:00:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.086+0000 7f87355b6640 1 -- 172.21.15.119:0/690159018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87300d1030 msgr2=0x7f87300c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.086+0000 7f87355b6640 1 --2- 172.21.15.119:0/690159018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87300d1030 0x7f87300c8f20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f872802acc0 tx=0x7f8728004a40 comp rx=0 tx=0).stop 2024-05-11T01:00:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.086+0000 7f87355b6640 1 -- 172.21.15.119:0/690159018 shutdown_connections 2024-05-11T01:00:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.086+0000 7f87355b6640 1 --2- 172.21.15.119:0/690159018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87300d1030 0x7f87300c8f20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.086+0000 7f87355b6640 1 --2- 172.21.15.119:0/690159018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87300cf940 0x7f8730153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.086+0000 7f87355b6640 1 --2- 172.21.15.119:0/690159018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300ceca0 0x7f873015b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.086+0000 7f87355b6640 1 -- 172.21.15.119:0/690159018 >> 172.21.15.119:0/690159018 conn(0x7f87300d2fa0 msgr2=0x7f873006da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.086+0000 7f87355b6640 1 -- 172.21.15.119:0/690159018 shutdown_connections 2024-05-11T01:00:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.086+0000 7f87355b6640 1 -- 172.21.15.119:0/690159018 wait complete. 2024-05-11T01:00:47.095 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:47.095+0000 7f741e410040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:00:47.097 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 3eba8bc2-884e-45a1-b06c-84e62a3300b0 7 2024-05-11T01:00:47.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.236+0000 7f53fb7e7640 1 Processor -- start 2024-05-11T01:00:47.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.236+0000 7f53fb7e7640 1 -- start start 2024-05-11T01:00:47.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53fb7e7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f40702d0 0x7f53f4166f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53fb7e7640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f415c520 con 0x7f53f406dca0 2024-05-11T01:00:47.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53fb7e7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f4071040 con 0x7f53f415c140 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53fb7e7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f53f406f680 con 0x7f53f40702d0 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53f9583640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f53f415c140 0x7f53f406d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52516/0 (socket says 172.21.15.119:52516) 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53f9d84640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f40702d0 0x7f53f4166f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53f9583640 1 -- 172.21.15.119:0/1346147660 learned_addr learned my addr 172.21.15.119:0/1346147660 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53ebfff640 1 -- 172.21.15.119:0/1346147660 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299072109 0 0) 0x7f53f4071040 con 0x7f53f415c140 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53ebfff640 1 -- 172.21.15.119:0/1346147660 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53e0005730 con 0x7f53f415c140 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53f9d84640 1 -- 172.21.15.119:0/1346147660 >> v1:172.21.15.119:6789/0 conn(0x7f53f415c140 legacy=0x7f53f406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53f9d84640 1 -- 172.21.15.119:0/1346147660 >> v1:172.21.15.52:6789/0 conn(0x7f53f406dca0 legacy=0x7f53f406fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.237+0000 7f53f9d84640 1 -- 172.21.15.119:0/1346147660 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f40707a0 con 0x7f53f40702d0 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.238+0000 7f53f9d84640 1 --2- 172.21.15.119:0/1346147660 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f40702d0 0x7f53f4166f60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009dc0 tx=0x7f53ec02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d8584fb9bdf56e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.238+0000 7f53ebfff640 1 -- 172.21.15.119:0/1346147660 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec007740 con 0x7f53f40702d0 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.238+0000 7f53ebfff640 1 -- 172.21.15.119:0/1346147660 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec0078e0 con 0x7f53f40702d0 2024-05-11T01:00:47.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.238+0000 7f53ebfff640 1 -- 172.21.15.119:0/1346147660 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec007cb0 con 0x7f53f40702d0 2024-05-11T01:00:47.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.239+0000 7f53fb7e7640 1 -- 172.21.15.119:0/1346147660 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f40702d0 msgr2=0x7f53f4166f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.239+0000 7f53fb7e7640 1 --2- 172.21.15.119:0/1346147660 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f40702d0 0x7f53f4166f60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009dc0 tx=0x7f53ec02a6d0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.239+0000 7f53fb7e7640 1 -- 172.21.15.119:0/1346147660 shutdown_connections 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.239+0000 7f53fb7e7640 1 --2- 172.21.15.119:0/1346147660 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f40702d0 0x7f53f4166f60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.239+0000 7f53fb7e7640 1 -- 172.21.15.119:0/1346147660 >> 172.21.15.119:0/1346147660 conn(0x7f53f40d1e50 msgr2=0x7f53f40d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.239+0000 7f53fb7e7640 1 -- 172.21.15.119:0/1346147660 shutdown_connections 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.239+0000 7f53fb7e7640 1 -- 172.21.15.119:0/1346147660 wait complete. 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.240+0000 7f53fb7e7640 1 Processor -- start 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.240+0000 7f53fb7e7640 1 -- start start 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53fb7e7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53f406dca0 0x7f53f40ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53fb7e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f40702d0 0x7f53f40ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53fb7e7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f415c140 0x7f53f40cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53fb7e7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f53f40cfac0 con 0x7f53f406dca0 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53fb7e7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4070d60 con 0x7f53f40702d0 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53fb7e7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f53f40ccc90 con 0x7f53f415c140 2024-05-11T01:00:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9d84640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f415c140 0x7f53f40cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f8d82640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f40702d0 0x7f53f40ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9d84640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f415c140 0x7f53f40cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:56504/0 (socket says 172.21.15.119:56504) 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9d84640 1 -- 172.21.15.119:0/3818044545 learned_addr learned my addr 172.21.15.119:0/3818044545 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9583640 1 --2- 172.21.15.119:0/3818044545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53f406dca0 0x7f53f40ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9d84640 1 -- 172.21.15.119:0/3818044545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f40702d0 msgr2=0x7f53f40ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9d84640 1 --2- 172.21.15.119:0/3818044545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f40702d0 0x7f53f40ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9d84640 1 -- 172.21.15.119:0/3818044545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53f406dca0 msgr2=0x7f53f40ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9d84640 1 --2- 172.21.15.119:0/3818044545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53f406dca0 0x7f53f40ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9d84640 1 -- 172.21.15.119:0/3818044545 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f40b51a0 con 0x7f53f415c140 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f8d82640 1 --2- 172.21.15.119:0/3818044545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f40702d0 0x7f53f40ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.241+0000 7f53f9d84640 1 --2- 172.21.15.119:0/3818044545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f415c140 0x7f53f40cb300 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009ee0 tx=0x7f53ec004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.242+0000 7f53e9ffb640 1 -- 172.21.15.119:0/3818044545 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec040070 con 0x7f53f415c140 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.242+0000 7f53e9ffb640 1 -- 172.21.15.119:0/3818044545 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec031bc0 con 0x7f53f415c140 2024-05-11T01:00:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.242+0000 7f53e9ffb640 1 -- 172.21.15.119:0/3818044545 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec009450 con 0x7f53f415c140 2024-05-11T01:00:47.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.242+0000 7f53fb7e7640 1 -- 172.21.15.119:0/3818044545 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f40b5480 con 0x7f53f415c140 2024-05-11T01:00:47.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.242+0000 7f53fb7e7640 1 -- 172.21.15.119:0/3818044545 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f40b5a40 con 0x7f53f415c140 2024-05-11T01:00:47.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.242+0000 7f53e9ffb640 1 -- 172.21.15.119:0/3818044545 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53ec039040 con 0x7f53f415c140 2024-05-11T01:00:47.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.242+0000 7f53e9ffb640 1 -- 172.21.15.119:0/3818044545 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f53ec045d20 con 0x7f53f415c140 2024-05-11T01:00:47.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.243+0000 7f53fb7e7640 1 -- 172.21.15.119:0/3818044545 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f4069c00 con 0x7f53f415c140 2024-05-11T01:00:47.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.245+0000 7f53e9ffb640 1 -- 172.21.15.119:0/3818044545 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f53ec007840 con 0x7f53f415c140 2024-05-11T01:00:47.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.317+0000 7f53fb7e7640 1 -- 172.21.15.119:0/3818044545 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3eba8bc2-884e-45a1-b06c-84e62a3300b0", "id": 7} v 0) -- 0x7f53f40b7dd0 con 0x7f53f415c140 2024-05-11T01:00:47.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.320+0000 7f53e9ffb640 1 -- 172.21.15.119:0/3818044545 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3eba8bc2-884e-45a1-b06c-84e62a3300b0", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f53ec045460 con 0x7f53f415c140 2024-05-11T01:00:47.320 INFO:teuthology.orchestra.run.smithi119.stdout:7 2024-05-11T01:00:47.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.321+0000 7f53fb7e7640 1 -- 172.21.15.119:0/3818044545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f415c140 msgr2=0x7f53f40cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.321+0000 7f53fb7e7640 1 --2- 172.21.15.119:0/3818044545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f415c140 0x7f53f40cb300 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009ee0 tx=0x7f53ec004a40 comp rx=0 tx=0).stop 2024-05-11T01:00:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.321+0000 7f53fb7e7640 1 -- 172.21.15.119:0/3818044545 shutdown_connections 2024-05-11T01:00:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.321+0000 7f53fb7e7640 1 --2- 172.21.15.119:0/3818044545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53f415c140 0x7f53f40cb300 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.321+0000 7f53fb7e7640 1 --2- 172.21.15.119:0/3818044545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f40702d0 0x7f53f40ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.321+0000 7f53fb7e7640 1 --2- 172.21.15.119:0/3818044545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53f406dca0 0x7f53f40ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.321+0000 7f53fb7e7640 1 -- 172.21.15.119:0/3818044545 >> 172.21.15.119:0/3818044545 conn(0x7f53f40d1e50 msgr2=0x7f53f40b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.321+0000 7f53fb7e7640 1 -- 172.21.15.119:0/3818044545 shutdown_connections 2024-05-11T01:00:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:00:47.321+0000 7f53fb7e7640 1 -- 172.21.15.119:0/3818044545 wait complete. 2024-05-11T01:00:47.332 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-11T01:00:47.332 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-11T01:00:47.334 INFO:tasks.ceph.osd.0:Started 2024-05-11T01:00:47.334 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-11T01:00:47.334 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-11T01:00:47.336 INFO:tasks.ceph.osd.1:Started 2024-05-11T01:00:47.336 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-11T01:00:47.337 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-11T01:00:47.339 INFO:tasks.ceph.osd.2:Started 2024-05-11T01:00:47.339 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-11T01:00:47.339 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-11T01:00:47.342 INFO:tasks.ceph.osd.3:Started 2024-05-11T01:00:47.342 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-11T01:00:47.342 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-11T01:00:47.344 INFO:tasks.ceph.osd.4:Started 2024-05-11T01:00:47.344 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-11T01:00:47.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-11T01:00:47.347 INFO:tasks.ceph.osd.5:Started 2024-05-11T01:00:47.347 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-11T01:00:47.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-11T01:00:47.349 INFO:tasks.ceph.osd.6:Started 2024-05-11T01:00:47.349 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-11T01:00:47.349 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-11T01:00:47.352 INFO:tasks.ceph.osd.7:Started 2024-05-11T01:00:47.352 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:00:47.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.491+0000 7fc1cc1c1640 1 Processor -- start 2024-05-11T01:00:47.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.491+0000 7fc1cc1c1640 1 -- start start 2024-05-11T01:00:47.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1cc1c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc1c40ced20 0x7fc1c40d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1cc1c1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40cf100 0x7fc1c4070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1cc1c1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1c4071220 0x7fc1c41630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1cc1c1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c40d2120 con 0x7fc1c4071220 2024-05-11T01:00:47.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1cc1c1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c40d1270 con 0x7fc1c40ced20 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1cc1c1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c40d25b0 con 0x7fc1c40cf100 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1ca75e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1c4071220 0x7fc1c41630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1ca75e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1c4071220 0x7fc1c41630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45532/0 (socket says 172.21.15.52:45532) 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1ca75e640 1 -- 172.21.15.52:0/3047109222 learned_addr learned my addr 172.21.15.52:0/3047109222 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1c9f5d640 1 --2- 172.21.15.52:0/3047109222 >> v2:172.21.15.119:3300/0 conn(0x7fc1c40ced20 0x7fc1c40d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1c975c640 1 --2- 172.21.15.52:0/3047109222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40cf100 0x7fc1c4070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1ca75e640 1 -- 172.21.15.52:0/3047109222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40cf100 msgr2=0x7fc1c4070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1ca75e640 1 --2- 172.21.15.52:0/3047109222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40cf100 0x7fc1c4070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1ca75e640 1 -- 172.21.15.52:0/3047109222 >> v2:172.21.15.119:3300/0 conn(0x7fc1c40ced20 msgr2=0x7fc1c40d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1ca75e640 1 --2- 172.21.15.52:0/3047109222 >> v2:172.21.15.119:3300/0 conn(0x7fc1c40ced20 0x7fc1c40d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.492+0000 7fc1ca75e640 1 -- 172.21.15.52:0/3047109222 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c40d1510 con 0x7fc1c4071220 2024-05-11T01:00:47.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.493+0000 7fc1ca75e640 1 --2- 172.21.15.52:0/3047109222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c4071220 0x7fc1c41630c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4073990 tx=0x7fc1bc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1484c831e2628f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:47.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.493+0000 7fc1c8f5b640 1 -- 172.21.15.52:0/3047109222 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc036070 con 0x7fc1c4071220 2024-05-11T01:00:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.493+0000 7fc1c8f5b640 1 -- 172.21.15.52:0/3047109222 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc031600 con 0x7fc1c4071220 2024-05-11T01:00:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.493+0000 7fc1c8f5b640 1 -- 172.21.15.52:0/3047109222 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0319d0 con 0x7fc1c4071220 2024-05-11T01:00:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.494+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/3047109222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c4071220 msgr2=0x7fc1c41630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.494+0000 7fc1cc1c1640 1 --2- 172.21.15.52:0/3047109222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c4071220 0x7fc1c41630c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4073990 tx=0x7fc1bc02a730 comp rx=0 tx=0).stop 2024-05-11T01:00:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.494+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/3047109222 shutdown_connections 2024-05-11T01:00:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.494+0000 7fc1cc1c1640 1 --2- 172.21.15.52:0/3047109222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c4071220 0x7fc1c41630c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.494+0000 7fc1cc1c1640 1 --2- 172.21.15.52:0/3047109222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40cf100 0x7fc1c4070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.494+0000 7fc1cc1c1640 1 --2- 172.21.15.52:0/3047109222 >> v2:172.21.15.119:3300/0 conn(0x7fc1c40ced20 0x7fc1c40d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.494+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/3047109222 >> 172.21.15.52:0/3047109222 conn(0x7fc1c40d2f00 msgr2=0x7fc1c40d3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.494+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/3047109222 shutdown_connections 2024-05-11T01:00:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.494+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/3047109222 wait complete. 2024-05-11T01:00:47.496 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:47.495+0000 7fe21c11c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.495+0000 7fc1cc1c1640 1 Processor -- start 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.495+0000 7fc1cc1c1640 1 -- start start 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1cc1c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c4071220 0x7fc1c409cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1cc1c1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40ced20 0x7fc1c414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1cc1c1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c40cf100 0x7fc1c40cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1cc1c1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40d1db0 con 0x7fc1c40cf100 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1cc1c1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40d1ad0 con 0x7fc1c4071220 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1cc1c1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c40707c0 con 0x7fc1c40ced20 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1ca75e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c40cf100 0x7fc1c40cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1ca75e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c40cf100 0x7fc1c40cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45536/0 (socket says 172.21.15.52:45536) 2024-05-11T01:00:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1ca75e640 1 -- 172.21.15.52:0/658366282 learned_addr learned my addr 172.21.15.52:0/658366282 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1c9f5d640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c4071220 0x7fc1c409cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1c975c640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40ced20 0x7fc1c414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1c9f5d640 1 -- 172.21.15.52:0/658366282 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40ced20 msgr2=0x7fc1c414ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1c9f5d640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40ced20 0x7fc1c414ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1c9f5d640 1 -- 172.21.15.52:0/658366282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c40cf100 msgr2=0x7fc1c40cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1c9f5d640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c40cf100 0x7fc1c40cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1c9f5d640 1 -- 172.21.15.52:0/658366282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c40cbf70 con 0x7fc1c4071220 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.496+0000 7fc1ca75e640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c40cf100 0x7fc1c40cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.497+0000 7fc1c9f5d640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c4071220 0x7fc1c409cde0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000e880 tx=0x7fc1c000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.497+0000 7fc1ba7fc640 1 -- 172.21.15.52:0/658366282 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0017070 con 0x7fc1c4071220 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.497+0000 7fc1ba7fc640 1 -- 172.21.15.52:0/658366282 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c000c940 con 0x7fc1c4071220 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.497+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/658366282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c40cc210 con 0x7fc1c4071220 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.497+0000 7fc1ba7fc640 1 -- 172.21.15.52:0/658366282 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000cd30 con 0x7fc1c4071220 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.497+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/658366282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c40cc7d0 con 0x7fc1c4071220 2024-05-11T01:00:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.498+0000 7fc1ba7fc640 1 -- 172.21.15.52:0/658366282 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc1c001a070 con 0x7fc1c4071220 2024-05-11T01:00:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.498+0000 7fc1ba7fc640 1 -- 172.21.15.52:0/658366282 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc1c0006680 con 0x7fc1c4071220 2024-05-11T01:00:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.498+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/658366282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c4069780 con 0x7fc1c4071220 2024-05-11T01:00:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.502+0000 7fc1ba7fc640 1 -- 172.21.15.52:0/658366282 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc1c0005990 con 0x7fc1c4071220 2024-05-11T01:00:47.521 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:47.521+0000 7f741e410040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T01:00:47.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.564+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/658366282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1c4162e30 con 0x7fc1c4071220 2024-05-11T01:00:47.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.565+0000 7fc1ba7fc640 1 -- 172.21.15.52:0/658366282 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc1c0005b70 con 0x7fc1c4071220 2024-05-11T01:00:47.565 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:47.565 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":10,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:00:47.319302+0000","last_up_change":"0.000000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:00:47.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.566+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/658366282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c4071220 msgr2=0x7fc1c409cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:47.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.566+0000 7fc1cc1c1640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c4071220 0x7fc1c409cde0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000e880 tx=0x7fc1c000ed40 comp rx=0 tx=0).stop 2024-05-11T01:00:47.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.566+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/658366282 shutdown_connections 2024-05-11T01:00:47.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.566+0000 7fc1cc1c1640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1c40cf100 0x7fc1c40cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.566+0000 7fc1cc1c1640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc1c40ced20 0x7fc1c414ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.566+0000 7fc1cc1c1640 1 --2- 172.21.15.52:0/658366282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c4071220 0x7fc1c409cde0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:47.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.566+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/658366282 >> 172.21.15.52:0/658366282 conn(0x7fc1c40d2f00 msgr2=0x7fc1c406df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:47.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.566+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/658366282 shutdown_connections 2024-05-11T01:00:47.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:47.566+0000 7fc1cc1c1640 1 -- 172.21.15.52:0/658366282 wait complete. 2024-05-11T01:00:47.578 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-11T01:00:47.578 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-11T01:00:47.644 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:47.644+0000 7fe21c11c040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T01:00:47.659 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:47.659+0000 7f741e410040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T01:00:47.916 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:47.917+0000 7fe21c11c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T01:00:47.918 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:47.918+0000 7f741e410040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T01:00:47.941 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-05-11T01:00:47.941+0000 7ff67a618640 -1 Falling back to public interface 2024-05-11T01:00:47.947 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-11T01:00:47.947+0000 7f4e41528640 -1 Falling back to public interface 2024-05-11T01:00:47.950 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-05-11T01:00:47.950+0000 7f21d60df640 -1 Falling back to public interface 2024-05-11T01:00:47.956 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-11T01:00:47.956+0000 7ff895a86640 -1 Falling back to public interface 2024-05-11T01:00:47.962 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-05-11T01:00:47.962+0000 7f22c5c18640 -1 Falling back to public interface 2024-05-11T01:00:47.964 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-11T01:00:47.964+0000 7f809fa0c640 -1 Falling back to public interface 2024-05-11T01:00:47.971 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-11T01:00:47.971+0000 7fdfefb57640 -1 Falling back to public interface 2024-05-11T01:00:47.971 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:47.972+0000 7f741e410040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T01:00:47.972 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:47.972+0000 7fe21c11c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T01:00:47.979 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-05-11T01:00:47.979+0000 7f15c0f01640 -1 Falling back to public interface 2024-05-11T01:00:48.093 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:48.093+0000 7f741e410040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T01:00:48.095 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:48.095+0000 7fe21c11c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T01:00:48.385 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T01:00:48.395 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:48.395+0000 7f741e410040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T01:00:48.400 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:48.400+0000 7fe21c11c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T01:00:48.444 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:48.444+0000 7f741e410040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T01:00:48.452 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:48.452+0000 7fe21c11c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T01:00:48.481 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.481+0000 7f8f81771640 1 Processor -- start 2024-05-11T01:00:48.482 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.481+0000 7f8f81771640 1 -- start start 2024-05-11T01:00:48.483 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.482+0000 7f8f81771640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 0x7f8f7c0d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.482+0000 7f8f81771640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7c0b82c0 con 0x7f8f7c0b66d0 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.482+0000 7f8f81771640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7c0b85a0 con 0x7f8f7c15ee70 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.482+0000 7f8f81771640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c0d41a0 con 0x7f8f7c0d3800 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.482+0000 7f8f7affd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8f7c0b66d0 0x7f8f7c0d0940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:34212/0 (socket says 172.21.15.52:34212) 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.482+0000 7f8f7affd640 1 -- 172.21.15.52:0/3543253591 learned_addr learned my addr 172.21.15.52:0/3543253591 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.482+0000 7f8f7a7fc640 1 --2- 172.21.15.52:0/3543253591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 0x7f8f7c0d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.482+0000 7f8f79ffb640 1 -- 172.21.15.52:0/3543253591 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2936040444 0 0) 0x7f8f7c0b82c0 con 0x7f8f7c0b66d0 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.482+0000 7f8f79ffb640 1 -- 172.21.15.52:0/3543253591 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f68003520 con 0x7f8f7c0b66d0 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.483+0000 7f8f79ffb640 1 -- 172.21.15.52:0/3543253591 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437826278 0 0) 0x7f8f7c0b85a0 con 0x7f8f7c15ee70 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.483+0000 7f8f79ffb640 1 -- 172.21.15.52:0/3543253591 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f7c0b82c0 con 0x7f8f7c15ee70 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.483+0000 7f8f7a7fc640 1 -- 172.21.15.52:0/3543253591 >> v1:172.21.15.119:6789/0 conn(0x7f8f7c15ee70 legacy=0x7f8f7c15f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.483+0000 7f8f7a7fc640 1 -- 172.21.15.52:0/3543253591 >> v1:172.21.15.52:6789/0 conn(0x7f8f7c0b66d0 legacy=0x7f8f7c0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:48.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.483+0000 7f8f7a7fc640 1 -- 172.21.15.52:0/3543253591 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c0b8880 con 0x7f8f7c0d3800 2024-05-11T01:00:48.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.483+0000 7f8f7a7fc640 1 --2- 172.21.15.52:0/3543253591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 0x7f8f7c0d3c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8f640097d0 tx=0x7f8f6402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be9c530597aee272 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:48.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f79ffb640 1 -- 172.21.15.52:0/3543253591 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64037070 con 0x7f8f7c0d3800 2024-05-11T01:00:48.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f79ffb640 1 -- 172.21.15.52:0/3543253591 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f64032440 con 0x7f8f7c0d3800 2024-05-11T01:00:48.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f79ffb640 1 -- 172.21.15.52:0/3543253591 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640327f0 con 0x7f8f7c0d3800 2024-05-11T01:00:48.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f81771640 1 -- 172.21.15.52:0/3543253591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 msgr2=0x7f8f7c0d3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:48.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f81771640 1 --2- 172.21.15.52:0/3543253591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 0x7f8f7c0d3c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8f640097d0 tx=0x7f8f6402a210 comp rx=0 tx=0).stop 2024-05-11T01:00:48.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f81771640 1 -- 172.21.15.52:0/3543253591 shutdown_connections 2024-05-11T01:00:48.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f81771640 1 --2- 172.21.15.52:0/3543253591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 0x7f8f7c0d3c60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:48.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f81771640 1 -- 172.21.15.52:0/3543253591 >> 172.21.15.52:0/3543253591 conn(0x7f8f7c09e570 msgr2=0x7f8f7c0d5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:48.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f81771640 1 -- 172.21.15.52:0/3543253591 shutdown_connections 2024-05-11T01:00:48.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.484+0000 7f8f81771640 1 -- 172.21.15.52:0/3543253591 wait complete. 2024-05-11T01:00:48.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.486+0000 7f8f81771640 1 Processor -- start 2024-05-11T01:00:48.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.486+0000 7f8f81771640 1 -- start start 2024-05-11T01:00:48.489 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.486+0000 7f8f81771640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7c0b66d0 0x7f8f7c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:48.489 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.486+0000 7f8f81771640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 0x7f8f7c159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:48.489 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.486+0000 7f8f81771640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f7c15ee70 0x7f8f7c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:48.489 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.486+0000 7f8f81771640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0b6ab0 con 0x7f8f7c15ee70 2024-05-11T01:00:48.489 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.486+0000 7f8f81771640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0b8070 con 0x7f8f7c0b66d0 2024-05-11T01:00:48.489 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.486+0000 7f8f81771640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c0d1050 con 0x7f8f7c0d3800 2024-05-11T01:00:48.489 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f7c15ee70 0x7f8f7c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:48.489 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7c0b66d0 0x7f8f7c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:48.489 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f7c15ee70 0x7f8f7c1615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45552/0 (socket says 172.21.15.52:45552) 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7b7fe640 1 -- 172.21.15.52:0/1859034162 learned_addr learned my addr 172.21.15.52:0/1859034162 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7a7fc640 1 --2- 172.21.15.52:0/1859034162 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 0x7f8f7c159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7b7fe640 1 -- 172.21.15.52:0/1859034162 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 msgr2=0x7f8f7c159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7b7fe640 1 --2- 172.21.15.52:0/1859034162 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 0x7f8f7c159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7b7fe640 1 -- 172.21.15.52:0/1859034162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7c0b66d0 msgr2=0x7f8f7c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7b7fe640 1 --2- 172.21.15.52:0/1859034162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7c0b66d0 0x7f8f7c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7b7fe640 1 -- 172.21.15.52:0/1859034162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c1612e0 con 0x7f8f7c15ee70 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f7b7fe640 1 --2- 172.21.15.52:0/1859034162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f7c15ee70 0x7f8f7c1615f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c7b0 tx=0x7f8f7400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f737fe640 1 -- 172.21.15.52:0/1859034162 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74003b70 con 0x7f8f7c15ee70 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f737fe640 1 -- 172.21.15.52:0/1859034162 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f74003d10 con 0x7f8f7c15ee70 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f81771640 1 -- 172.21.15.52:0/1859034162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c0c9990 con 0x7f8f7c15ee70 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f737fe640 1 -- 172.21.15.52:0/1859034162 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400aa90 con 0x7f8f7c15ee70 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.487+0000 7f8f81771640 1 -- 172.21.15.52:0/1859034162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c0c9f50 con 0x7f8f7c15ee70 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.488+0000 7f8f737fe640 1 -- 172.21.15.52:0/1859034162 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8f7400ac30 con 0x7f8f7c15ee70 2024-05-11T01:00:48.490 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.488+0000 7f8f737fe640 1 -- 172.21.15.52:0/1859034162 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f8f74012c80 con 0x7f8f7c15ee70 2024-05-11T01:00:48.491 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.488+0000 7f8f81771640 1 -- 172.21.15.52:0/1859034162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c069fa0 con 0x7f8f7c15ee70 2024-05-11T01:00:48.491 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.490+0000 7f8f737fe640 1 -- 172.21.15.52:0/1859034162 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8f740125b0 con 0x7f8f7c15ee70 2024-05-11T01:00:48.553 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.553+0000 7f8f81771640 1 -- 172.21.15.52:0/1859034162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f7c068ef0 con 0x7f8f7c15ee70 2024-05-11T01:00:48.554 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.554+0000 7f8f737fe640 1 -- 172.21.15.52:0/1859034162 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f8f74012790 con 0x7f8f7c15ee70 2024-05-11T01:00:48.554 INFO:teuthology.misc.health.smithi052.stdout: 2024-05-11T01:00:48.554 INFO:teuthology.misc.health.smithi052.stdout:{"epoch":10,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:00:47.319302+0000","last_up_change":"0.000000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:00:48.555 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.555+0000 7f8f81771640 1 -- 172.21.15.52:0/1859034162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f7c15ee70 msgr2=0x7f8f7c1615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:48.555 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.555+0000 7f8f81771640 1 --2- 172.21.15.52:0/1859034162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f7c15ee70 0x7f8f7c1615f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c7b0 tx=0x7f8f7400cc70 comp rx=0 tx=0).stop 2024-05-11T01:00:48.556 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.555+0000 7f8f81771640 1 -- 172.21.15.52:0/1859034162 shutdown_connections 2024-05-11T01:00:48.556 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.555+0000 7f8f81771640 1 --2- 172.21.15.52:0/1859034162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f7c15ee70 0x7f8f7c1615f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:48.556 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.555+0000 7f8f81771640 1 --2- 172.21.15.52:0/1859034162 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8f7c0d3800 0x7f8f7c159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:48.556 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.555+0000 7f8f81771640 1 --2- 172.21.15.52:0/1859034162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f7c0b66d0 0x7f8f7c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:48.556 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.555+0000 7f8f81771640 1 -- 172.21.15.52:0/1859034162 >> 172.21.15.52:0/1859034162 conn(0x7f8f7c09e570 msgr2=0x7f8f7c06d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:48.556 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.555+0000 7f8f81771640 1 -- 172.21.15.52:0/1859034162 shutdown_connections 2024-05-11T01:00:48.556 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:48.555+0000 7f8f81771640 1 -- 172.21.15.52:0/1859034162 wait complete. 2024-05-11T01:00:48.565 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-11T01:00:48.602 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:48.602+0000 7f741e410040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T01:00:48.620 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:48.620+0000 7fe21c11c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T01:00:48.736 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:48.736+0000 7f741e410040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T01:00:48.764 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:48.764+0000 7fe21c11c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T01:00:48.787 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:48.787+0000 7f741e410040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T01:00:48.817 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:48.817+0000 7fe21c11c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T01:00:48.859 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:48.858+0000 7f741e410040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T01:00:48.890 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:48.890+0000 7fe21c11c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T01:00:48.903 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:48.903+0000 7f741e410040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T01:00:48.938 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:48.938+0000 7fe21c11c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T01:00:49.385 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:49.383+0000 7f741e410040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T01:00:49.440 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:49.440+0000 7fe21c11c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T01:00:49.498 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:49.498+0000 7f741e410040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T01:00:49.566 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:00:49.566+0000 7fe21c11c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T01:00:50.513 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:50.513+0000 7f73bda8e640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-11T01:00:50.513 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:00:50.513+0000 7f73bda8e640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-11T01:00:51.407 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-05-11T01:00:51.407+0000 7ff67a618640 -1 osd.4 0 log_to_monitors true 2024-05-11T01:00:51.409 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-11T01:00:51.409+0000 7f4e41528640 -1 osd.0 0 log_to_monitors true 2024-05-11T01:00:51.437 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-05-11T01:00:51.437+0000 7f21d60df640 -1 osd.6 0 log_to_monitors true 2024-05-11T01:00:51.456 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-11T01:00:51.456+0000 7ff895a86640 -1 osd.1 0 log_to_monitors true 2024-05-11T01:00:51.477 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-11T01:00:51.477+0000 7f809fa0c640 -1 osd.3 0 log_to_monitors true 2024-05-11T01:00:51.484 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-11T01:00:51.484+0000 7fdfefb57640 -1 osd.2 0 log_to_monitors true 2024-05-11T01:00:51.487 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-05-11T01:00:51.487+0000 7f22c5c18640 -1 osd.7 0 log_to_monitors true 2024-05-11T01:00:51.496 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-05-11T01:00:51.497+0000 7f15c0f01640 -1 osd.5 0 log_to_monitors true 2024-05-11T01:00:53.929 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-11T01:00:53.928+0000 7ff891a1c640 -1 osd.1 0 waiting for initial osdmap 2024-05-11T01:00:53.944 INFO:tasks.ceph.osd.1.smithi052.stderr:2024-05-11T01:00:53.944+0000 7ff88983c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T01:00:54.012 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-11T01:00:54.012+0000 7fdfebaed640 -1 osd.2 0 waiting for initial osdmap 2024-05-11T01:00:54.022 INFO:tasks.ceph.osd.2.smithi052.stderr:2024-05-11T01:00:54.022+0000 7fdfe390d640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T01:00:54.031 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-11T01:00:54.032+0000 7f809bbe2640 -1 osd.3 0 waiting for initial osdmap 2024-05-11T01:00:54.043 INFO:tasks.ceph.osd.3.smithi052.stderr:2024-05-11T01:00:54.043+0000 7f80931ee640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T01:00:54.056 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-11T01:00:54.056+0000 7f4e3d4be640 -1 osd.0 0 waiting for initial osdmap 2024-05-11T01:00:54.065 INFO:tasks.ceph.osd.0.smithi052.stderr:2024-05-11T01:00:54.066+0000 7f4e352de640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T01:00:54.530 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-05-11T01:00:54.529+0000 7f22c1580640 -1 osd.7 0 waiting for initial osdmap 2024-05-11T01:00:54.545 INFO:tasks.ceph.osd.7.smithi119.stderr:2024-05-11T01:00:54.545+0000 7f22b93a0640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T01:00:54.570 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-05-11T01:00:54.570+0000 7ff675fe7640 -1 osd.4 0 waiting for initial osdmap 2024-05-11T01:00:54.584 INFO:tasks.ceph.osd.4.smithi119.stderr:2024-05-11T01:00:54.584+0000 7ff66de07640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T01:00:54.588 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-05-11T01:00:54.588+0000 7f15bce97640 -1 osd.5 0 waiting for initial osdmap 2024-05-11T01:00:54.599 INFO:tasks.ceph.osd.5.smithi119.stderr:2024-05-11T01:00:54.599+0000 7f15b4cb7640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T01:00:54.630 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-05-11T01:00:54.630+0000 7f21d2075640 -1 osd.6 0 waiting for initial osdmap 2024-05-11T01:00:54.654 INFO:tasks.ceph.osd.6.smithi119.stderr:2024-05-11T01:00:54.654+0000 7f21c9e95640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T01:00:55.372 DEBUG:teuthology.orchestra.run.smithi052:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T01:00:55.479 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.479+0000 7f51ac699640 1 Processor -- start 2024-05-11T01:00:55.480 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.479+0000 7f51ac699640 1 -- start start 2024-05-11T01:00:55.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.480+0000 7f51ac699640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51a40cff60 0x7f51a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.480+0000 7f51ac699640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a406dff0 con 0x7f51a406d530 2024-05-11T01:00:55.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.480+0000 7f51ac699640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a406e2d0 con 0x7f51a415f400 2024-05-11T01:00:55.484 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.480+0000 7f51ac699640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f51a4070c20 con 0x7f51a40cff60 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9c34640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f51a406d530 0x7f51a40cf850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54448/0 (socket says 172.21.15.52:54448) 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9c34640 1 -- 172.21.15.52:0/610969886 learned_addr learned my addr 172.21.15.52:0/610969886 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51aac36640 1 --2- 172.21.15.52:0/610969886 >> v2:172.21.15.119:3300/0 conn(0x7f51a40cff60 0x7f51a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1372318862 0 0) 0x7f51a406dff0 con 0x7f51a406d530 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5190003520 con 0x7f51a406d530 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3300115405 0 0) 0x7f51a406e2d0 con 0x7f51a415f400 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51a406dff0 con 0x7f51a415f400 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1834798299 0 0) 0x7f5190003520 con 0x7f51a406d530 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 >> v2:172.21.15.119:3300/0 conn(0x7f51a40cff60 msgr2=0x7f51a406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 --2- 172.21.15.52:0/610969886 >> v2:172.21.15.119:3300/0 conn(0x7f51a40cff60 0x7f51a406d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.485 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 >> v1:172.21.15.119:6789/0 conn(0x7f51a415f400 legacy=0x7f51a4070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a406e5b0 con 0x7f51a406d530 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f51940016f0 con 0x7f51a406d530 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 >> v1:172.21.15.52:6789/0 conn(0x7f51a406d530 legacy=0x7f51a40cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5190003c30 0x7f5190004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5190004800 0x7f5190004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5190005210 0x7f519000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4070c20 con 0x7f5190005210 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f519000c140 con 0x7f5190004800 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f519000c340 con 0x7f5190003c30 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51aac36640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5190005210 0x7f519000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51a9c34640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5190004800 0x7f5190004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.486 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.481+0000 7f51aa435640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5190003c30 0x7f5190004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.482+0000 7f51aac36640 1 -- 172.21.15.52:0/610969886 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5190003c30 msgr2=0x7f5190004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.482+0000 7f51aac36640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5190003c30 0x7f5190004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.482+0000 7f51aac36640 1 -- 172.21.15.52:0/610969886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5190004800 msgr2=0x7f5190004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.482+0000 7f51aac36640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5190004800 0x7f5190004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.482+0000 7f51aac36640 1 -- 172.21.15.52:0/610969886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5190003520 con 0x7f5190005210 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.482+0000 7f51aac36640 1 -- 172.21.15.52:0/610969886 reap_dead start 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.482+0000 7f51aac36640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5190005210 0x7f519000b9c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51a40d3fd0 tx=0x7f5190005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ceed76bb547ff8e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c009c10 con 0x7f5190005210 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519c031410 con 0x7f5190005210 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51a9433640 1 -- 172.21.15.52:0/610969886 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c0317e0 con 0x7f5190005210 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51ac699640 1 -- 172.21.15.52:0/610969886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5190005210 msgr2=0x7f519000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.487 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51ac699640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5190005210 0x7f519000b9c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51a40d3fd0 tx=0x7f5190005830 comp rx=0 tx=0).stop 2024-05-11T01:00:55.488 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51ac699640 1 -- 172.21.15.52:0/610969886 shutdown_connections 2024-05-11T01:00:55.488 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51ac699640 1 --2- 172.21.15.52:0/610969886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5190005210 0x7f519000b9c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.488 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51ac699640 1 -- 172.21.15.52:0/610969886 >> 172.21.15.52:0/610969886 conn(0x7f51a40d2fd0 msgr2=0x7f51a40d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:55.488 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51ac699640 1 -- 172.21.15.52:0/610969886 shutdown_connections 2024-05-11T01:00:55.488 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.483+0000 7f51ac699640 1 -- 172.21.15.52:0/610969886 wait complete. 2024-05-11T01:00:55.491 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.485+0000 7f51ac699640 1 Processor -- start 2024-05-11T01:00:55.491 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.485+0000 7f51ac699640 1 -- start start 2024-05-11T01:00:55.491 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.485+0000 7f51ac699640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5190005210 0x7f51a4150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.491 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.485+0000 7f51ac699640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51a41f53d0 0x7f51a40c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.491 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.485+0000 7f51ac699640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51a40b0d90 0x7f51a40b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.491 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.485+0000 7f51ac699640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f51a415fbb0 con 0x7f51a40b0d90 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.485+0000 7f51ac699640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f51a415f7e0 con 0x7f5190005210 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.485+0000 7f51ac699640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f51a40d1020 con 0x7f51a41f53d0 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aac36640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51a40b0d90 0x7f51a40b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aac36640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51a40b0d90 0x7f51a40b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52438/0 (socket says 172.21.15.52:52438) 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aac36640 1 -- 172.21.15.52:0/3735813101 learned_addr learned my addr 172.21.15.52:0/3735813101 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aa435640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5190005210 0x7f51a4150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51a9c34640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51a41f53d0 0x7f51a40c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aac36640 1 -- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51a41f53d0 msgr2=0x7f51a40c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aac36640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51a41f53d0 0x7f51a40c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aac36640 1 -- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5190005210 msgr2=0x7f51a4150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aac36640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5190005210 0x7f51a4150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aac36640 1 -- 172.21.15.52:0/3735813101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a41f5910 con 0x7f51a40b0d90 2024-05-11T01:00:55.492 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51aac36640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51a40b0d90 0x7f51a40b1210 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f519c02aeb0 tx=0x7f519c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f519affd640 1 -- 172.21.15.52:0/3735813101 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c009c10 con 0x7f51a40b0d90 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f519affd640 1 -- 172.21.15.52:0/3735813101 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519c043070 con 0x7f51a40b0d90 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f519affd640 1 -- 172.21.15.52:0/3735813101 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c03c820 con 0x7f51a40b0d90 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a41f5bf0 con 0x7f51a40b0d90 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.486+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41f61b0 con 0x7f51a40b0d90 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.488+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a4069fa0 con 0x7f51a40b0d90 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.488+0000 7f519affd640 1 -- 172.21.15.52:0/3735813101 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f519c005d10 con 0x7f51a40b0d90 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.488+0000 7f519affd640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5188076240 0x7f5188078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.489+0000 7f519affd640 1 -- 172.21.15.52:0/3735813101 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4191+0+0 (secure 0 0 0) 0x7f519c0b7760 con 0x7f51a40b0d90 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.489+0000 7f51aa435640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5188076240 0x7f5188078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.493 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.489+0000 7f51aa435640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5188076240 0x7f5188078700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f51a0001870 tx=0x7f51a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:55.494 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.493+0000 7f519affd640 1 -- 172.21.15.52:0/3735813101 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f519c02e050 con 0x7f51a40b0d90 2024-05-11T01:00:55.692 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.692+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51a4161b50 con 0x7f51a40b0d90 2024-05-11T01:00:55.692 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.693+0000 7f519affd640 1 -- 172.21.15.52:0/3735813101 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+12865 (secure 0 0 0) 0x7f519c081790 con 0x7f51a40b0d90 2024-05-11T01:00:55.693 INFO:teuthology.misc.health.smithi052.stdout: 2024-05-11T01:00:55.693 INFO:teuthology.misc.health.smithi052.stdout:{"epoch":14,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:00:55.632733+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:00:55.695 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5188076240 msgr2=0x7f5188078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.695 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5188076240 0x7f5188078700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f51a0001870 tx=0x7f51a0009040 comp rx=0 tx=0).stop 2024-05-11T01:00:55.695 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51a40b0d90 msgr2=0x7f51a40b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.695 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51a40b0d90 0x7f51a40b1210 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f519c02aeb0 tx=0x7f519c009750 comp rx=0 tx=0).stop 2024-05-11T01:00:55.695 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 shutdown_connections 2024-05-11T01:00:55.695 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5188076240 0x7f5188078700 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.695 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51a40b0d90 0x7f51a40b1210 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.695 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51a41f53d0 0x7f51a40c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.696 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 --2- 172.21.15.52:0/3735813101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5190005210 0x7f51a4150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.696 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 >> 172.21.15.52:0/3735813101 conn(0x7f51a40d2fd0 msgr2=0x7f51a4161390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:55.696 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 shutdown_connections 2024-05-11T01:00:55.696 INFO:teuthology.misc.health.smithi052.stderr:2024-05-11T01:00:55.695+0000 7f51ac699640 1 -- 172.21.15.52:0/3735813101 wait complete. 2024-05-11T01:00:55.705 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-11T01:00:55.705 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-11T01:00:55.705 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T01:00:55.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T01:00:55.707 INFO:tasks.ceph.mds.a:Started 2024-05-11T01:00:55.707 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T01:00:55.708 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T01:00:55.709 INFO:tasks.ceph.mds.c:Started 2024-05-11T01:00:55.710 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T01:00:55.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T01:00:55.712 INFO:tasks.ceph.mds.b:Started 2024-05-11T01:00:55.712 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T01:00:55.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T01:00:55.715 INFO:tasks.ceph.mds.d:Started 2024-05-11T01:00:55.715 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-11T01:00:55.715 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:00:55.786 INFO:tasks.ceph.mds.c.smithi052.stdout:starting mds.c at 2024-05-11T01:00:55.787 INFO:tasks.ceph.mds.a.smithi052.stdout:starting mds.a at 2024-05-11T01:00:55.799 INFO:tasks.ceph.mds.b.smithi119.stdout:starting mds.b at 2024-05-11T01:00:55.799 INFO:tasks.ceph.mds.d.smithi119.stdout:starting mds.d at 2024-05-11T01:00:55.799 INFO:tasks.ceph.mds.d.smithi119.stderr:2024-05-11T01:00:55.799+0000 7f643126e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-11T01:00:55.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.824+0000 7ff306946640 1 Processor -- start 2024-05-11T01:00:55.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.824+0000 7ff306946640 1 -- start start 2024-05-11T01:00:55.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff306946640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff30009c130 0x7ff3000b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff306946640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff300160e00 0x7ff3000b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff306946640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3000d1000 con 0x7ff300160a20 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff306946640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff3000d20a0 con 0x7ff300160e00 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff306946640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff3000d12e0 con 0x7ff30009c130 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff304ee3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff300160e00 0x7ff3000b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff2fffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff300160a20 0x7ff30009ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:38870/0 (socket says 172.21.15.52:38870) 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff2f7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff30009c130 0x7ff3000b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff2fffff640 1 -- 172.21.15.52:0/4148977832 learned_addr learned my addr 172.21.15.52:0/4148977832 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff304ee3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff300160e00 0x7ff3000b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52472/0 (socket says 172.21.15.52:52472) 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff304ee3640 1 -- 172.21.15.52:0/4148977832 >> v2:172.21.15.119:3300/0 conn(0x7ff30009c130 msgr2=0x7ff3000b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff304ee3640 1 --2- 172.21.15.52:0/4148977832 >> v2:172.21.15.119:3300/0 conn(0x7ff30009c130 0x7ff3000b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff304ee3640 1 -- 172.21.15.52:0/4148977832 >> v1:172.21.15.119:6789/0 conn(0x7ff300160a20 legacy=0x7ff30009ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff304ee3640 1 -- 172.21.15.52:0/4148977832 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3000d15c0 con 0x7ff300160e00 2024-05-11T01:00:55.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.825+0000 7ff2f7fff640 1 --2- 172.21.15.52:0/4148977832 >> v2:172.21.15.119:3300/0 conn(0x7ff30009c130 0x7ff3000b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.826+0000 7ff304ee3640 1 --2- 172.21.15.52:0/4148977832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160e00 0x7ff3000b55b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff30006cf60 tx=0x7ff2f802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e821daa86cb85ca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.826+0000 7ff2ff7fe640 1 -- 172.21.15.52:0/4148977832 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8007740 con 0x7ff300160e00 2024-05-11T01:00:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.826+0000 7ff2ff7fe640 1 -- 172.21.15.52:0/4148977832 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f80078e0 con 0x7ff300160e00 2024-05-11T01:00:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.826+0000 7ff2ff7fe640 1 -- 172.21.15.52:0/4148977832 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8007cb0 con 0x7ff300160e00 2024-05-11T01:00:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.827+0000 7ff306946640 1 -- 172.21.15.52:0/4148977832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160e00 msgr2=0x7ff3000b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.827+0000 7ff306946640 1 --2- 172.21.15.52:0/4148977832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160e00 0x7ff3000b55b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff30006cf60 tx=0x7ff2f802a680 comp rx=0 tx=0).stop 2024-05-11T01:00:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.827+0000 7ff306946640 1 -- 172.21.15.52:0/4148977832 shutdown_connections 2024-05-11T01:00:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.827+0000 7ff306946640 1 --2- 172.21.15.52:0/4148977832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160e00 0x7ff3000b55b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.827+0000 7ff306946640 1 --2- 172.21.15.52:0/4148977832 >> v2:172.21.15.119:3300/0 conn(0x7ff30009c130 0x7ff3000b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.827+0000 7ff306946640 1 -- 172.21.15.52:0/4148977832 >> 172.21.15.52:0/4148977832 conn(0x7ff3000b4a90 msgr2=0x7ff3000b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.827+0000 7ff306946640 1 -- 172.21.15.52:0/4148977832 shutdown_connections 2024-05-11T01:00:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.827+0000 7ff306946640 1 -- 172.21.15.52:0/4148977832 wait complete. 2024-05-11T01:00:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.828+0000 7ff306946640 1 Processor -- start 2024-05-11T01:00:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.828+0000 7ff306946640 1 -- start start 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff306946640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff30009c130 0x7ff3000ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff306946640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160a20 0x7ff3000cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff306946640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff300160e00 0x7ff30015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff306946640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000cdb70 con 0x7ff300160a20 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff306946640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000d1b80 con 0x7ff30009c130 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff306946640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff3000cdcf0 con 0x7ff300160e00 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2f7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160a20 0x7ff3000cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2f7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160a20 0x7ff3000cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52488/0 (socket says 172.21.15.52:52488) 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2f7fff640 1 -- 172.21.15.52:0/527774818 learned_addr learned my addr 172.21.15.52:0/527774818 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff304ee3640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff300160e00 0x7ff30015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2fffff640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff30009c130 0x7ff3000ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2f7fff640 1 -- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff300160e00 msgr2=0x7ff30015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2f7fff640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff300160e00 0x7ff30015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2f7fff640 1 -- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff30009c130 msgr2=0x7ff3000ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2f7fff640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff30009c130 0x7ff3000ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2f7fff640 1 -- 172.21.15.52:0/527774818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3000caf40 con 0x7ff300160a20 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.829+0000 7ff2f7fff640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160a20 0x7ff3000cc930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800ebe0 tx=0x7ff2e800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.830+0000 7ff2fd7fa640 1 -- 172.21.15.52:0/527774818 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8017070 con 0x7ff300160a20 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.830+0000 7ff2fd7fa640 1 -- 172.21.15.52:0/527774818 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e800cc30 con 0x7ff300160a20 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.830+0000 7ff2fd7fa640 1 -- 172.21.15.52:0/527774818 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8013470 con 0x7ff300160a20 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.830+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30006e180 con 0x7ff300160a20 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.830+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30006e740 con 0x7ff300160a20 2024-05-11T01:00:55.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.831+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff300069780 con 0x7ff300160a20 2024-05-11T01:00:55.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.831+0000 7ff2fd7fa640 1 -- 172.21.15.52:0/527774818 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7ff2e801a070 con 0x7ff300160a20 2024-05-11T01:00:55.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.832+0000 7ff2fd7fa640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2e0076240 0x7ff2e0078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:55.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.832+0000 7ff2fd7fa640 1 -- 172.21.15.52:0/527774818 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4846+0+0 (secure 0 0 0) 0x7ff2e8097470 con 0x7ff300160a20 2024-05-11T01:00:55.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.832+0000 7ff2fffff640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2e0076240 0x7ff2e0078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:55.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.833+0000 7ff2fffff640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2e0076240 0x7ff2e0078700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0000f60 tx=0x7ff2f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:55.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:55.837+0000 7ff2fd7fa640 1 -- 172.21.15.52:0/527774818 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff2e8061210 con 0x7ff300160a20 2024-05-11T01:00:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.068+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3000b2ea0 con 0x7ff300160a20 2024-05-11T01:00:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.068+0000 7ff2fd7fa640 1 -- 172.21.15.52:0/527774818 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7ff2e8069070 con 0x7ff300160a20 2024-05-11T01:00:56.069 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:56.069 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-11T01:00:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 2 2024-05-11T01:00:56.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2e0076240 msgr2=0x7ff2e0078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:56.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2e0076240 0x7ff2e0078700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0000f60 tx=0x7ff2f0009040 comp rx=0 tx=0).stop 2024-05-11T01:00:56.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160a20 msgr2=0x7ff3000cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:56.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160a20 0x7ff3000cc930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800ebe0 tx=0x7ff2e800c520 comp rx=0 tx=0).stop 2024-05-11T01:00:56.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 shutdown_connections 2024-05-11T01:00:56.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2e0076240 0x7ff2e0078700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:56.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff300160e00 0x7ff30015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff300160a20 0x7ff3000cc930 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 --2- 172.21.15.52:0/527774818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff30009c130 0x7ff3000ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 >> 172.21.15.52:0/527774818 conn(0x7ff3000b4a90 msgr2=0x7ff30015f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 shutdown_connections 2024-05-11T01:00:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:56.071+0000 7ff306946640 1 -- 172.21.15.52:0/527774818 wait complete. 2024-05-11T01:00:58.083 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:00:58.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.190+0000 7fc130de4640 1 Processor -- start 2024-05-11T01:00:58.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.190+0000 7fc130de4640 1 -- start start 2024-05-11T01:00:58.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc130de4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc12c160480 0x7fc12c0d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc130de4640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12c163250 con 0x7fc12c15f430 2024-05-11T01:00:58.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc130de4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12c163530 con 0x7fc12c0d32c0 2024-05-11T01:00:58.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc130de4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc12c163810 con 0x7fc12c160480 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc12a59c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc12c15f430 0x7fc12c15fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54478/0 (socket says 172.21.15.52:54478) 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc12a59c640 1 -- 172.21.15.52:0/3633221204 learned_addr learned my addr 172.21.15.52:0/3633221204 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc129d9b640 1 --2- 172.21.15.52:0/3633221204 >> v2:172.21.15.119:3300/0 conn(0x7fc12c160480 0x7fc12c0d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc12959a640 1 -- 172.21.15.52:0/3633221204 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1916314001 0 0) 0x7fc12c163250 con 0x7fc12c15f430 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc12959a640 1 -- 172.21.15.52:0/3633221204 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc114003520 con 0x7fc12c15f430 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc12959a640 1 -- 172.21.15.52:0/3633221204 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860633364 0 0) 0x7fc12c163530 con 0x7fc12c0d32c0 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc12959a640 1 -- 172.21.15.52:0/3633221204 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc12c163250 con 0x7fc12c0d32c0 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc129d9b640 1 -- 172.21.15.52:0/3633221204 >> v1:172.21.15.119:6789/0 conn(0x7fc12c0d32c0 legacy=0x7fc12c0d36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc129d9b640 1 -- 172.21.15.52:0/3633221204 >> v1:172.21.15.52:6789/0 conn(0x7fc12c15f430 legacy=0x7fc12c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.191+0000 7fc129d9b640 1 -- 172.21.15.52:0/3633221204 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c163af0 con 0x7fc12c160480 2024-05-11T01:00:58.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.192+0000 7fc129d9b640 1 --2- 172.21.15.52:0/3633221204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 0x7fc12c0d2d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc1200097a0 tx=0x7fc12002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eadf19a60eb4de0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:58.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.192+0000 7fc12959a640 1 -- 172.21.15.52:0/3633221204 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120037070 con 0x7fc12c160480 2024-05-11T01:00:58.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.192+0000 7fc12959a640 1 -- 172.21.15.52:0/3633221204 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc120032440 con 0x7fc12c160480 2024-05-11T01:00:58.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.192+0000 7fc12959a640 1 -- 172.21.15.52:0/3633221204 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120032810 con 0x7fc12c160480 2024-05-11T01:00:58.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.193+0000 7fc130de4640 1 -- 172.21.15.52:0/3633221204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 msgr2=0x7fc12c0d2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.193+0000 7fc130de4640 1 --2- 172.21.15.52:0/3633221204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 0x7fc12c0d2d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc1200097a0 tx=0x7fc12002a1a0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.193+0000 7fc130de4640 1 -- 172.21.15.52:0/3633221204 shutdown_connections 2024-05-11T01:00:58.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.193+0000 7fc130de4640 1 --2- 172.21.15.52:0/3633221204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 0x7fc12c0d2d80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.193+0000 7fc130de4640 1 -- 172.21.15.52:0/3633221204 >> 172.21.15.52:0/3633221204 conn(0x7fc12c08eb70 msgr2=0x7fc12c09e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:58.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.193+0000 7fc130de4640 1 -- 172.21.15.52:0/3633221204 shutdown_connections 2024-05-11T01:00:58.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.193+0000 7fc130de4640 1 -- 172.21.15.52:0/3633221204 wait complete. 2024-05-11T01:00:58.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.194+0000 7fc130de4640 1 Processor -- start 2024-05-11T01:00:58.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.194+0000 7fc130de4640 1 -- start start 2024-05-11T01:00:58.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc130de4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc12c0d32c0 0x7fc12c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc130de4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc12c15f430 0x7fc12c0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc130de4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 0x7fc12c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc130de4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c163030 con 0x7fc12c15f430 2024-05-11T01:00:58.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc130de4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c162eb0 con 0x7fc12c160480 2024-05-11T01:00:58.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc130de4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c1608f0 con 0x7fc12c0d32c0 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc129d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc12c15f430 0x7fc12c0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc129d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc12c15f430 0x7fc12c0b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52504/0 (socket says 172.21.15.52:52504) 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc129d9b640 1 -- 172.21.15.52:0/2025154185 learned_addr learned my addr 172.21.15.52:0/2025154185 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc12a59c640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc12c0d32c0 0x7fc12c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.195+0000 7fc12ad9d640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 0x7fc12c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.196+0000 7fc129d9b640 1 -- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc12c0d32c0 msgr2=0x7fc12c157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.196+0000 7fc129d9b640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc12c0d32c0 0x7fc12c157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.196+0000 7fc129d9b640 1 -- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 msgr2=0x7fc12c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.196+0000 7fc129d9b640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 0x7fc12c0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.196+0000 7fc129d9b640 1 -- 172.21.15.52:0/2025154185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c0cdfc0 con 0x7fc12c15f430 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.196+0000 7fc12ad9d640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 0x7fc12c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:58.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.196+0000 7fc129d9b640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc12c15f430 0x7fc12c0b2e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc12002a1e0 tx=0x7fc1200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:58.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.197+0000 7fc11affd640 1 -- 172.21.15.52:0/2025154185 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120037040 con 0x7fc12c15f430 2024-05-11T01:00:58.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.197+0000 7fc11affd640 1 -- 172.21.15.52:0/2025154185 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc120042070 con 0x7fc12c15f430 2024-05-11T01:00:58.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.197+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c0c9960 con 0x7fc12c15f430 2024-05-11T01:00:58.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.197+0000 7fc11affd640 1 -- 172.21.15.52:0/2025154185 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12003b930 con 0x7fc12c15f430 2024-05-11T01:00:58.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.197+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c0ce540 con 0x7fc12c15f430 2024-05-11T01:00:58.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.198+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12c0b42f0 con 0x7fc12c15f430 2024-05-11T01:00:58.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.198+0000 7fc11affd640 1 -- 172.21.15.52:0/2025154185 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7fc12003bad0 con 0x7fc12c15f430 2024-05-11T01:00:58.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.199+0000 7fc11affd640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc10c076240 0x7fc10c078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.199+0000 7fc11affd640 1 -- 172.21.15.52:0/2025154185 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc1200b7920 con 0x7fc12c15f430 2024-05-11T01:00:58.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.199+0000 7fc12a59c640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc10c076240 0x7fc10c078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.200+0000 7fc12a59c640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc10c076240 0x7fc10c078700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc11c005880 tx=0x7fc11c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:58.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.204+0000 7fc11affd640 1 -- 172.21.15.52:0/2025154185 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc12c0b42f0 con 0x7fc12c15f430 2024-05-11T01:00:58.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.208+0000 7fc11affd640 1 -- 172.21.15.52:0/2025154185 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc12007d2f0 con 0x7fc12c15f430 2024-05-11T01:00:58.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.436+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc12c06ef50 con 0x7fc12c15f430 2024-05-11T01:00:58.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.437+0000 7fc11affd640 1 -- 172.21.15.52:0/2025154185 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7fc1200b3560 con 0x7fc12c15f430 2024-05-11T01:00:58.437 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:58.437 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2245862597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2245862597},{"type":"v1","addr":"172.21.15.119:6837","nonce":2245862597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4286578298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4286578298},{"type":"v1","addr":"172.21.15.52:6835","nonce":4286578298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1484114523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1484114523},{"type":"v1","addr":"172.21.15.119:6836","nonce":1484114523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-11T01:00:58.438 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2024-05-11T01:00:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc10c076240 msgr2=0x7fc10c078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc10c076240 0x7fc10c078700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc11c005880 tx=0x7fc11c009040 comp rx=0 tx=0).stop 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc12c15f430 msgr2=0x7fc12c0b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc12c15f430 0x7fc12c0b2e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc12002a1e0 tx=0x7fc1200046c0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 shutdown_connections 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc10c076240 0x7fc10c078700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c160480 0x7fc12c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc12c15f430 0x7fc12c0b2e40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 --2- 172.21.15.52:0/2025154185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc12c0d32c0 0x7fc12c157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 >> 172.21.15.52:0/2025154185 conn(0x7fc12c08eb70 msgr2=0x7fc12c06e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 shutdown_connections 2024-05-11T01:00:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.439+0000 7fc130de4640 1 -- 172.21.15.52:0/2025154185 wait complete. 2024-05-11T01:00:58.450 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T01:00:58.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.554+0000 7f7f5d2fc640 1 Processor -- start 2024-05-11T01:00:58.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.554+0000 7f7f5d2fc640 1 -- start start 2024-05-11T01:00:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.555+0000 7f7f5d2fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7f580ce340 0x7f7f5806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.555+0000 7f7f5d2fc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f5806c9c0 0x7f7f580cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.555+0000 7f7f5d2fc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f58160e60 con 0x7f7f5815d400 2024-05-11T01:00:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.555+0000 7f7f5d2fc640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7f580ce7a0 con 0x7f7f580ce340 2024-05-11T01:00:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.555+0000 7f7f5d2fc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7f58161100 con 0x7f7f5806c9c0 2024-05-11T01:00:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.555+0000 7f7f567fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7f580ce340 0x7f7f5806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.555+0000 7f7f567fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7f580ce340 0x7f7f5806c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52506/0 (socket says 172.21.15.52:52506) 2024-05-11T01:00:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.555+0000 7f7f567fc640 1 -- 172.21.15.52:0/1470088818 learned_addr learned my addr 172.21.15.52:0/1470088818 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.556+0000 7f7f577fe640 1 --2- 172.21.15.52:0/1470088818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f5806c9c0 0x7f7f580cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.556+0000 7f7f567fc640 1 -- 172.21.15.52:0/1470088818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f5806c9c0 msgr2=0x7f7f580cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.556+0000 7f7f567fc640 1 --2- 172.21.15.52:0/1470088818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f5806c9c0 0x7f7f580cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.556+0000 7f7f567fc640 1 -- 172.21.15.52:0/1470088818 >> v1:172.21.15.119:6789/0 conn(0x7f7f5815d400 legacy=0x7f7f580b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.556+0000 7f7f567fc640 1 -- 172.21.15.52:0/1470088818 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f581613e0 con 0x7f7f580ce340 2024-05-11T01:00:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.556+0000 7f7f567fc640 1 --2- 172.21.15.52:0/1470088818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f580ce340 0x7f7f5806c480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009e50 tx=0x7f7f4002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363f269bff03b809 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:58.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f55ffb640 1 -- 172.21.15.52:0/1470088818 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40036070 con 0x7f7f580ce340 2024-05-11T01:00:58.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f55ffb640 1 -- 172.21.15.52:0/1470088818 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f40031600 con 0x7f7f580ce340 2024-05-11T01:00:58.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f55ffb640 1 -- 172.21.15.52:0/1470088818 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400319d0 con 0x7f7f580ce340 2024-05-11T01:00:58.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/1470088818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f580ce340 msgr2=0x7f7f5806c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f5d2fc640 1 --2- 172.21.15.52:0/1470088818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f580ce340 0x7f7f5806c480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009e50 tx=0x7f7f4002a710 comp rx=0 tx=0).stop 2024-05-11T01:00:58.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/1470088818 shutdown_connections 2024-05-11T01:00:58.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f5d2fc640 1 --2- 172.21.15.52:0/1470088818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f5806c9c0 0x7f7f580cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f5d2fc640 1 --2- 172.21.15.52:0/1470088818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f580ce340 0x7f7f5806c480 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/1470088818 >> 172.21.15.52:0/1470088818 conn(0x7f7f580b3a30 msgr2=0x7f7f580b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:58.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/1470088818 shutdown_connections 2024-05-11T01:00:58.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.557+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/1470088818 wait complete. 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.559+0000 7f7f5d2fc640 1 Processor -- start 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.559+0000 7f7f5d2fc640 1 -- start start 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.559+0000 7f7f5d2fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5806c9c0 0x7f7f58151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.559+0000 7f7f5d2fc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f580ce340 0x7f7f580cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.559+0000 7f7f5d2fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f5815d400 0x7f7f580d1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.559+0000 7f7f5d2fc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58161be0 con 0x7f7f5806c9c0 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.559+0000 7f7f5d2fc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7f580b1600 con 0x7f7f5815d400 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.559+0000 7f7f5d2fc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7f580ccce0 con 0x7f7f580ce340 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.559+0000 7f7f56ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5806c9c0 0x7f7f58151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f56ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5806c9c0 0x7f7f58151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52510/0 (socket says 172.21.15.52:52510) 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f56ffd640 1 -- 172.21.15.52:0/910860217 learned_addr learned my addr 172.21.15.52:0/910860217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f567fc640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f580ce340 0x7f7f580cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f577fe640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f5815d400 0x7f7f580d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f56ffd640 1 -- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f580ce340 msgr2=0x7f7f580cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f56ffd640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f580ce340 0x7f7f580cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f56ffd640 1 -- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f5815d400 msgr2=0x7f7f580d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f56ffd640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f5815d400 0x7f7f580d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f56ffd640 1 -- 172.21.15.52:0/910860217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f580d03f0 con 0x7f7f5806c9c0 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f577fe640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f5815d400 0x7f7f580d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f56ffd640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5806c9c0 0x7f7f58151440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c3a0 tx=0x7f7f4800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f4f7fe640 1 -- 172.21.15.52:0/910860217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48017070 con 0x7f7f5806c9c0 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f4f7fe640 1 -- 172.21.15.52:0/910860217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f480052d0 con 0x7f7f5806c9c0 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f580d06d0 con 0x7f7f5806c9c0 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f4f7fe640 1 -- 172.21.15.52:0/910860217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48003830 con 0x7f7f5806c9c0 2024-05-11T01:00:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.560+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f580d0c90 con 0x7f7f5806c9c0 2024-05-11T01:00:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.561+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f58069670 con 0x7f7f5806c9c0 2024-05-11T01:00:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.562+0000 7f7f4f7fe640 1 -- 172.21.15.52:0/910860217 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7f4801a070 con 0x7f7f5806c9c0 2024-05-11T01:00:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.562+0000 7f7f4f7fe640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7f380762d0 0x7f7f38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.563+0000 7f7f4f7fe640 1 -- 172.21.15.52:0/910860217 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7f48096560 con 0x7f7f5806c9c0 2024-05-11T01:00:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.563+0000 7f7f567fc640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7f380762d0 0x7f7f38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.563+0000 7f7f567fc640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7f380762d0 0x7f7f38078790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f40033000 tx=0x7f7f40009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:58.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.567+0000 7f7f4f7fe640 1 -- 172.21.15.52:0/910860217 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7f4805fbc0 con 0x7f7f5806c9c0 2024-05-11T01:00:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.759+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7f580b7ec0 con 0x7f7f5806c9c0 2024-05-11T01:00:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.759+0000 7f7f4f7fe640 1 -- 172.21.15.52:0/910860217 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f7f48068020 con 0x7f7f5806c9c0 2024-05-11T01:00:58.760 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:00:58.760 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-05-11T01:00:58.760 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:00:58.760 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-05-11T01:00:58.760 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-05-11T01:00:58.760 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:00:58.760 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7f380762d0 msgr2=0x7f7f38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7f380762d0 0x7f7f38078790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f40033000 tx=0x7f7f40009000 comp rx=0 tx=0).stop 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5806c9c0 msgr2=0x7f7f58151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5806c9c0 0x7f7f58151440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c3a0 tx=0x7f7f4800c860 comp rx=0 tx=0).stop 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 shutdown_connections 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7f380762d0 0x7f7f38078790 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f5815d400 0x7f7f580d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7f580ce340 0x7f7f580cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 --2- 172.21.15.52:0/910860217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5806c9c0 0x7f7f58151440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 >> 172.21.15.52:0/910860217 conn(0x7f7f580b3a30 msgr2=0x7f7f580b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 shutdown_connections 2024-05-11T01:00:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.761+0000 7f7f5d2fc640 1 -- 172.21.15.52:0/910860217 wait complete. 2024-05-11T01:00:58.772 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T01:00:58.773 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T01:00:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.880+0000 7fa4f7a93640 1 Processor -- start 2024-05-11T01:00:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.880+0000 7fa4f7a93640 1 -- start start 2024-05-11T01:00:58.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f7a93640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa4f015f310 0x7fa4f00d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f7a93640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa4f00b4dc0 0x7fa4f00d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f7a93640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f00cfd40 con 0x7fa4f00d55d0 2024-05-11T01:00:58.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f7a93640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f00b6140 con 0x7fa4f00b4dc0 2024-05-11T01:00:58.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f7a93640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f00d59e0 con 0x7fa4f015f310 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f502e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa4f00b4dc0 0x7fa4f00d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f502e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa4f00b4dc0 0x7fa4f00d5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52518/0 (socket says 172.21.15.52:52518) 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f502e640 1 -- 172.21.15.52:0/853376100 learned_addr learned my addr 172.21.15.52:0/853376100 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f582f640 1 --2- 172.21.15.52:0/853376100 >> v2:172.21.15.119:3300/0 conn(0x7fa4f015f310 0x7fa4f00d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f502e640 1 -- 172.21.15.52:0/853376100 >> v2:172.21.15.119:3300/0 conn(0x7fa4f015f310 msgr2=0x7fa4f00d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f502e640 1 --2- 172.21.15.52:0/853376100 >> v2:172.21.15.119:3300/0 conn(0x7fa4f015f310 0x7fa4f00d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f502e640 1 -- 172.21.15.52:0/853376100 >> v1:172.21.15.119:6789/0 conn(0x7fa4f00d55d0 legacy=0x7fa4f0162ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.881+0000 7fa4f502e640 1 -- 172.21.15.52:0/853376100 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f00b5530 con 0x7fa4f00b4dc0 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.882+0000 7fa4f502e640 1 --2- 172.21.15.52:0/853376100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f00b4dc0 0x7fa4f00d5090 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4f008e890 tx=0x7fa4e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2854f373214becb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.882+0000 7fa4f482d640 1 -- 172.21.15.52:0/853376100 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0037070 con 0x7fa4f00b4dc0 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.882+0000 7fa4f482d640 1 -- 172.21.15.52:0/853376100 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e0032440 con 0x7fa4f00b4dc0 2024-05-11T01:00:58.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.882+0000 7fa4f482d640 1 -- 172.21.15.52:0/853376100 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0032810 con 0x7fa4f00b4dc0 2024-05-11T01:00:58.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.883+0000 7fa4f7a93640 1 -- 172.21.15.52:0/853376100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f00b4dc0 msgr2=0x7fa4f00d5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.883+0000 7fa4f7a93640 1 --2- 172.21.15.52:0/853376100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f00b4dc0 0x7fa4f00d5090 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4f008e890 tx=0x7fa4e002a270 comp rx=0 tx=0).stop 2024-05-11T01:00:58.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.883+0000 7fa4f7a93640 1 -- 172.21.15.52:0/853376100 shutdown_connections 2024-05-11T01:00:58.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.883+0000 7fa4f7a93640 1 --2- 172.21.15.52:0/853376100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f00b4dc0 0x7fa4f00d5090 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.883+0000 7fa4f7a93640 1 --2- 172.21.15.52:0/853376100 >> v2:172.21.15.119:3300/0 conn(0x7fa4f015f310 0x7fa4f00d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.883+0000 7fa4f7a93640 1 -- 172.21.15.52:0/853376100 >> 172.21.15.52:0/853376100 conn(0x7fa4f008eb80 msgr2=0x7fa4f009e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:58.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.883+0000 7fa4f7a93640 1 -- 172.21.15.52:0/853376100 shutdown_connections 2024-05-11T01:00:58.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.883+0000 7fa4f7a93640 1 -- 172.21.15.52:0/853376100 wait complete. 2024-05-11T01:00:58.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.884+0000 7fa4f7a93640 1 Processor -- start 2024-05-11T01:00:58.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.884+0000 7fa4f7a93640 1 -- start start 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f7a93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4f00b4dc0 0x7fa4f015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f7a93640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa4f00d55d0 0x7fa4f015fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f7a93640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f015f310 0x7fa4f0160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f7a93640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f00b5dd0 con 0x7fa4f015f310 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f7a93640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f00b5af0 con 0x7fa4f00b4dc0 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f7a93640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f00b5250 con 0x7fa4f00d55d0 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f6030640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f015f310 0x7fa4f0160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f6030640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f015f310 0x7fa4f0160320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52526/0 (socket says 172.21.15.52:52526) 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f6030640 1 -- 172.21.15.52:0/3709187291 learned_addr learned my addr 172.21.15.52:0/3709187291 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f582f640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4f00b4dc0 0x7fa4f015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f502e640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa4f00d55d0 0x7fa4f015fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f6030640 1 -- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa4f00d55d0 msgr2=0x7fa4f015fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f6030640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa4f00d55d0 0x7fa4f015fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f6030640 1 -- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4f00b4dc0 msgr2=0x7fa4f015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f6030640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4f00b4dc0 0x7fa4f015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f6030640 1 -- 172.21.15.52:0/3709187291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f00c9a40 con 0x7fa4f015f310 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f502e640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa4f00d55d0 0x7fa4f015fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.885+0000 7fa4f6030640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f015f310 0x7fa4f0160320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c7a0 tx=0x7fa4e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.886+0000 7fa4e67fc640 1 -- 172.21.15.52:0/3709187291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8003b70 con 0x7fa4f015f310 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.886+0000 7fa4e67fc640 1 -- 172.21.15.52:0/3709187291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e8003d10 con 0x7fa4f015f310 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.886+0000 7fa4e67fc640 1 -- 172.21.15.52:0/3709187291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e800aa70 con 0x7fa4f015f310 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.886+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f00c9d20 con 0x7fa4f015f310 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.886+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f00ca2e0 con 0x7fa4f015f310 2024-05-11T01:00:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.887+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f00d49d0 con 0x7fa4f015f310 2024-05-11T01:00:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.887+0000 7fa4e67fc640 1 -- 172.21.15.52:0/3709187291 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa4e800ac10 con 0x7fa4f015f310 2024-05-11T01:00:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.888+0000 7fa4e67fc640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa4d40762d0 0x7fa4d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.888+0000 7fa4e67fc640 1 -- 172.21.15.52:0/3709187291 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa4e80970c0 con 0x7fa4f015f310 2024-05-11T01:00:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.888+0000 7fa4f582f640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa4d40762d0 0x7fa4d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.888+0000 7fa4f582f640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa4d40762d0 0x7fa4d4078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006fd0 tx=0x7fa4ec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:58.892+0000 7fa4e67fc640 1 -- 172.21.15.52:0/3709187291 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa4f00d49d0 con 0x7fa4f015f310 2024-05-11T01:00:59.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.082+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa4f00b4330 con 0x7fa4f015f310 2024-05-11T01:00:59.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.660+0000 7fa4e67fc640 1 -- 172.21.15.52:0/3709187291 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fa4f00cfd40 con 0x7fa4f015f310 2024-05-11T01:00:59.660 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-05-11T01:00:59.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.664+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa4d40762d0 msgr2=0x7fa4d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:59.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.664+0000 7fa4f7a93640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa4d40762d0 0x7fa4d4078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006fd0 tx=0x7fa4ec008040 comp rx=0 tx=0).stop 2024-05-11T01:00:59.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.664+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f015f310 msgr2=0x7fa4f0160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:59.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.664+0000 7fa4f7a93640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f015f310 0x7fa4f0160320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c7a0 tx=0x7fa4e800cc60 comp rx=0 tx=0).stop 2024-05-11T01:00:59.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.665+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 shutdown_connections 2024-05-11T01:00:59.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.665+0000 7fa4f7a93640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa4d40762d0 0x7fa4d4078790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:59.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.665+0000 7fa4f7a93640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4f015f310 0x7fa4f0160320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:59.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.665+0000 7fa4f7a93640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa4f00d55d0 0x7fa4f015fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:59.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.665+0000 7fa4f7a93640 1 --2- 172.21.15.52:0/3709187291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4f00b4dc0 0x7fa4f015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:59.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.665+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 >> 172.21.15.52:0/3709187291 conn(0x7fa4f008eb80 msgr2=0x7fa4f00d1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:59.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.666+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 shutdown_connections 2024-05-11T01:00:59.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.667+0000 7fa4f7a93640 1 -- 172.21.15.52:0/3709187291 wait complete. 2024-05-11T01:00:59.682 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T01:00:59.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.799+0000 7fb6f6d96640 1 Processor -- start 2024-05-11T01:00:59.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.799+0000 7fb6f6d96640 1 -- start start 2024-05-11T01:00:59.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f6d96640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00d2330 0x7fb6f016a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:59.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f6d96640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f00d12d0 con 0x7fb6f015f6f0 2024-05-11T01:00:59.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f6d96640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f00d1570 con 0x7fb6f006d630 2024-05-11T01:00:59.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f6d96640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f006dc10 con 0x7fb6f00d2330 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f4b32640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb6f015f6f0 0x7fb6f00cd650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54488/0 (socket says 172.21.15.52:54488) 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f4b32640 1 -- 172.21.15.52:0/821838566 learned_addr learned my addr 172.21.15.52:0/821838566 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f5333640 1 --2- 172.21.15.52:0/821838566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00d2330 0x7fb6f016a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6ef7fe640 1 -- 172.21.15.52:0/821838566 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748221799 0 0) 0x7fb6f00d12d0 con 0x7fb6f015f6f0 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6ef7fe640 1 -- 172.21.15.52:0/821838566 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6e0003520 con 0x7fb6f015f6f0 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6ef7fe640 1 -- 172.21.15.52:0/821838566 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1056779195 0 0) 0x7fb6f00d1570 con 0x7fb6f006d630 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6ef7fe640 1 -- 172.21.15.52:0/821838566 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6f00d12d0 con 0x7fb6f006d630 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f5333640 1 -- 172.21.15.52:0/821838566 >> v1:172.21.15.119:6789/0 conn(0x7fb6f006d630 legacy=0x7fb6f00cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f5333640 1 -- 172.21.15.52:0/821838566 >> v1:172.21.15.52:6789/0 conn(0x7fb6f015f6f0 legacy=0x7fb6f00cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:59.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.800+0000 7fb6f5333640 1 -- 172.21.15.52:0/821838566 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f00d1850 con 0x7fb6f00d2330 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.801+0000 7fb6f5333640 1 --2- 172.21.15.52:0/821838566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00d2330 0x7fb6f016a460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009df0 tx=0x7fb6e802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9a9c5b28b4048bb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.801+0000 7fb6ef7fe640 1 -- 172.21.15.52:0/821838566 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8007740 con 0x7fb6f00d2330 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.801+0000 7fb6ef7fe640 1 -- 172.21.15.52:0/821838566 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e80078e0 con 0x7fb6f00d2330 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.801+0000 7fb6ef7fe640 1 -- 172.21.15.52:0/821838566 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8007c90 con 0x7fb6f00d2330 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.802+0000 7fb6f6d96640 1 -- 172.21.15.52:0/821838566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00d2330 msgr2=0x7fb6f016a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.802+0000 7fb6f6d96640 1 --2- 172.21.15.52:0/821838566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00d2330 0x7fb6f016a460 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009df0 tx=0x7fb6e802a710 comp rx=0 tx=0).stop 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.802+0000 7fb6f6d96640 1 -- 172.21.15.52:0/821838566 shutdown_connections 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.802+0000 7fb6f6d96640 1 --2- 172.21.15.52:0/821838566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00d2330 0x7fb6f016a460 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.802+0000 7fb6f6d96640 1 -- 172.21.15.52:0/821838566 >> 172.21.15.52:0/821838566 conn(0x7fb6f00d3000 msgr2=0x7fb6f00d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.802+0000 7fb6f6d96640 1 -- 172.21.15.52:0/821838566 shutdown_connections 2024-05-11T01:00:59.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.802+0000 7fb6f6d96640 1 -- 172.21.15.52:0/821838566 wait complete. 2024-05-11T01:00:59.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.803+0000 7fb6f6d96640 1 Processor -- start 2024-05-11T01:00:59.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.803+0000 7fb6f6d96640 1 -- start start 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f6d96640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f006d630 0x7fb6f00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f6d96640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6f00d2330 0x7fb6f015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f6d96640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00cb4c0 0x7fb6f0155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f6d96640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00d3b90 con 0x7fb6f006d630 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f6d96640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00d1e10 con 0x7fb6f00d2330 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f6d96640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f016a9a0 con 0x7fb6f00cb4c0 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f4b32640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f006d630 0x7fb6f00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f4b32640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f006d630 0x7fb6f00cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52540/0 (socket says 172.21.15.52:52540) 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f4b32640 1 -- 172.21.15.52:0/1356919523 learned_addr learned my addr 172.21.15.52:0/1356919523 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6effff640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6f00d2330 0x7fb6f015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:59.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f5333640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00cb4c0 0x7fb6f0155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6effff640 1 -- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00cb4c0 msgr2=0x7fb6f0155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6effff640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00cb4c0 0x7fb6f0155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6effff640 1 -- 172.21.15.52:0/1356919523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f006d630 msgr2=0x7fb6f00cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6effff640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f006d630 0x7fb6f00cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6effff640 1 -- 172.21.15.52:0/1356919523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f00ca420 con 0x7fb6f00d2330 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.804+0000 7fb6f4b32640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f006d630 0x7fb6f00cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.805+0000 7fb6effff640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6f00d2330 0x7fb6f015afb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00c440 tx=0x7fb6dc00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.805+0000 7fb6ed7fa640 1 -- 172.21.15.52:0/1356919523 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0052d0 con 0x7fb6f00d2330 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.805+0000 7fb6ed7fa640 1 -- 172.21.15.52:0/1356919523 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc005490 con 0x7fb6f00d2330 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.805+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f00ca700 con 0x7fb6f00d2330 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.805+0000 7fb6ed7fa640 1 -- 172.21.15.52:0/1356919523 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc012870 con 0x7fb6f00d2330 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.805+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f00cacc0 con 0x7fb6f00d2330 2024-05-11T01:00:59.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.806+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f0069c00 con 0x7fb6f00d2330 2024-05-11T01:00:59.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.806+0000 7fb6ed7fa640 1 -- 172.21.15.52:0/1356919523 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb6dc012a10 con 0x7fb6f00d2330 2024-05-11T01:00:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.807+0000 7fb6ed7fa640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6d4076300 0x7fb6d40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:00:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.807+0000 7fb6ed7fa640 1 -- 172.21.15.52:0/1356919523 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fb6dc096e70 con 0x7fb6f00d2330 2024-05-11T01:00:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.807+0000 7fb6f4b32640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6d4076300 0x7fb6d40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:00:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.808+0000 7fb6f4b32640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6d4076300 0x7fb6d40787c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4000c00 tx=0x7fb6e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:00:59.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:00:59.812+0000 7fb6ed7fa640 1 -- 172.21.15.52:0/1356919523 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb6dc0602c0 con 0x7fb6f00d2330 2024-05-11T01:01:00.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.004+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb6f006e950 con 0x7fb6f00d2330 2024-05-11T01:01:00.671 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-05-11T01:01:00.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.670+0000 7fb6ed7fa640 1 -- 172.21.15.52:0/1356919523 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fb6dc0645c0 con 0x7fb6f00d2330 2024-05-11T01:01:00.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.678+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6d4076300 msgr2=0x7fb6d40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:00.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.678+0000 7fb6f6d96640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6d4076300 0x7fb6d40787c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4000c00 tx=0x7fb6e4009040 comp rx=0 tx=0).stop 2024-05-11T01:01:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.678+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6f00d2330 msgr2=0x7fb6f015afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:00.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.678+0000 7fb6f6d96640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6f00d2330 0x7fb6f015afb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00c440 tx=0x7fb6dc00c900 comp rx=0 tx=0).stop 2024-05-11T01:01:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.681+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 shutdown_connections 2024-05-11T01:01:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.681+0000 7fb6f6d96640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6d4076300 0x7fb6d40787c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.681+0000 7fb6f6d96640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb6f00cb4c0 0x7fb6f0155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.681+0000 7fb6f6d96640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6f00d2330 0x7fb6f015afb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.681+0000 7fb6f6d96640 1 --2- 172.21.15.52:0/1356919523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f006d630 0x7fb6f00cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.681+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 >> 172.21.15.52:0/1356919523 conn(0x7fb6f00d3000 msgr2=0x7fb6f00b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:00.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.681+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 shutdown_connections 2024-05-11T01:01:00.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.682+0000 7fb6f6d96640 1 -- 172.21.15.52:0/1356919523 wait complete. 2024-05-11T01:01:00.698 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T01:01:00.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.810+0000 7faac3196640 1 Processor -- start 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.810+0000 7faac3196640 1 -- start start 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faac3196640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faabc0d4e40 0x7faabc0d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faac3196640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faabc0d3a90 con 0x7faabc0d2d60 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faac3196640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faabc0d3d70 con 0x7faabc15f2f0 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faac3196640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7faabc0d4050 con 0x7faabc0d4e40 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faac1733640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7faabc0d2d60 0x7faabc0d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:54504/0 (socket says 172.21.15.52:54504) 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faac1733640 1 -- 172.21.15.52:0/4176902968 learned_addr learned my addr 172.21.15.52:0/4176902968 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faab3fff640 1 --2- 172.21.15.52:0/4176902968 >> v2:172.21.15.119:3300/0 conn(0x7faabc0d4e40 0x7faabc0d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faab37fe640 1 -- 172.21.15.52:0/4176902968 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728670701 0 0) 0x7faabc0d3a90 con 0x7faabc0d2d60 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faab37fe640 1 -- 172.21.15.52:0/4176902968 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaa4003520 con 0x7faabc0d2d60 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faab3fff640 1 -- 172.21.15.52:0/4176902968 >> v1:172.21.15.119:6789/0 conn(0x7faabc15f2f0 legacy=0x7faabc0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faab3fff640 1 -- 172.21.15.52:0/4176902968 >> v1:172.21.15.52:6789/0 conn(0x7faabc0d2d60 legacy=0x7faabc0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:00.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.811+0000 7faab3fff640 1 -- 172.21.15.52:0/4176902968 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc0b4e60 con 0x7faabc0d4e40 2024-05-11T01:01:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.812+0000 7faab3fff640 1 --2- 172.21.15.52:0/4176902968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faabc0d4e40 0x7faabc0d5290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faaac0097a0 tx=0x7faaac02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edc7c1b1f400ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.812+0000 7faab37fe640 1 -- 172.21.15.52:0/4176902968 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac037070 con 0x7faabc0d4e40 2024-05-11T01:01:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.812+0000 7faab37fe640 1 -- 172.21.15.52:0/4176902968 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaac032440 con 0x7faabc0d4e40 2024-05-11T01:01:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.812+0000 7faab37fe640 1 -- 172.21.15.52:0/4176902968 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0327f0 con 0x7faabc0d4e40 2024-05-11T01:01:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.813+0000 7faac3196640 1 -- 172.21.15.52:0/4176902968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faabc0d4e40 msgr2=0x7faabc0d5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.813+0000 7faac3196640 1 --2- 172.21.15.52:0/4176902968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faabc0d4e40 0x7faabc0d5290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faaac0097a0 tx=0x7faaac02a230 comp rx=0 tx=0).stop 2024-05-11T01:01:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.813+0000 7faac3196640 1 -- 172.21.15.52:0/4176902968 shutdown_connections 2024-05-11T01:01:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.813+0000 7faac3196640 1 --2- 172.21.15.52:0/4176902968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faabc0d4e40 0x7faabc0d5290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.813+0000 7faac3196640 1 -- 172.21.15.52:0/4176902968 >> 172.21.15.52:0/4176902968 conn(0x7faabc08ebb0 msgr2=0x7faabc09e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:00.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.813+0000 7faac3196640 1 -- 172.21.15.52:0/4176902968 shutdown_connections 2024-05-11T01:01:00.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.813+0000 7faac3196640 1 -- 172.21.15.52:0/4176902968 wait complete. 2024-05-11T01:01:00.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.814+0000 7faac3196640 1 Processor -- start 2024-05-11T01:01:00.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.814+0000 7faac3196640 1 -- start start 2024-05-11T01:01:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac3196640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faabc0d2d60 0x7faabc160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac3196640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faabc0d4e40 0x7faabc0cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac3196640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faabc15f2f0 0x7faabc0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac3196640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faabc15f6d0 con 0x7faabc0d2d60 2024-05-11T01:01:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac3196640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faabc0d57d0 con 0x7faabc15f2f0 2024-05-11T01:01:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac3196640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7faabc15f850 con 0x7faabc0d4e40 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac0f32640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faabc0d2d60 0x7faabc160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac0f32640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faabc0d2d60 0x7faabc160a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52546/0 (socket says 172.21.15.52:52546) 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac0f32640 1 -- 172.21.15.52:0/3963753802 learned_addr learned my addr 172.21.15.52:0/3963753802 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac1733640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faabc15f2f0 0x7faabc0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faab3fff640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faabc0d4e40 0x7faabc0cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac0f32640 1 -- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faabc0d4e40 msgr2=0x7faabc0cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac0f32640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faabc0d4e40 0x7faabc0cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac0f32640 1 -- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faabc15f2f0 msgr2=0x7faabc0cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac0f32640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faabc15f2f0 0x7faabc0cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.815+0000 7faac0f32640 1 -- 172.21.15.52:0/3963753802 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc1613c0 con 0x7faabc0d2d60 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.816+0000 7faac0f32640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faabc0d2d60 0x7faabc160a50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faab800c400 tx=0x7faab800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.816+0000 7faab17fa640 1 -- 172.21.15.52:0/3963753802 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8017070 con 0x7faabc0d2d60 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.816+0000 7faab17fa640 1 -- 172.21.15.52:0/3963753802 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab80052d0 con 0x7faabc0d2d60 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.816+0000 7faab17fa640 1 -- 172.21.15.52:0/3963753802 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8003830 con 0x7faabc0d2d60 2024-05-11T01:01:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.816+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc1616a0 con 0x7faabc0d2d60 2024-05-11T01:01:00.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.816+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc161c60 con 0x7faabc0d2d60 2024-05-11T01:01:00.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.817+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc0695d0 con 0x7faabc0d2d60 2024-05-11T01:01:00.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.817+0000 7faab17fa640 1 -- 172.21.15.52:0/3963753802 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faab801a070 con 0x7faabc0d2d60 2024-05-11T01:01:00.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.818+0000 7faab17fa640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7faaa80762d0 0x7faaa8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:00.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.818+0000 7faab17fa640 1 -- 172.21.15.52:0/3963753802 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7faab80968e0 con 0x7faabc0d2d60 2024-05-11T01:01:00.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.818+0000 7faab3fff640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7faaa80762d0 0x7faaa8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:00.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.819+0000 7faab3fff640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7faaa80762d0 0x7faaa8078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faaac034000 tx=0x7faaac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:00.823+0000 7faab17fa640 1 -- 172.21.15.52:0/3963753802 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faab805fc50 con 0x7faabc0d2d60 2024-05-11T01:01:01.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.051+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faabc0d6080 con 0x7faabc0d2d60 2024-05-11T01:01:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.683+0000 7faab17fa640 1 -- 172.21.15.52:0/3963753802 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7faab8068290 con 0x7faabc0d2d60 2024-05-11T01:01:01.688 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 2 and data pool 3 2024-05-11T01:01:01.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.696+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7faaa80762d0 msgr2=0x7faaa8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:01.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.696+0000 7faac3196640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7faaa80762d0 0x7faaa8078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faaac034000 tx=0x7faaac02e000 comp rx=0 tx=0).stop 2024-05-11T01:01:01.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.696+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faabc0d2d60 msgr2=0x7faabc160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:01.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.696+0000 7faac3196640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faabc0d2d60 0x7faabc160a50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faab800c400 tx=0x7faab800c8c0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.698+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 shutdown_connections 2024-05-11T01:01:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.698+0000 7faac3196640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7faaa80762d0 0x7faaa8078790 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.698+0000 7faac3196640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faabc15f2f0 0x7faabc0cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.698+0000 7faac3196640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faabc0d4e40 0x7faabc0cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.698+0000 7faac3196640 1 --2- 172.21.15.52:0/3963753802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faabc0d2d60 0x7faabc160a50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.698+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 >> 172.21.15.52:0/3963753802 conn(0x7faabc08ebb0 msgr2=0x7faabc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.698+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 shutdown_connections 2024-05-11T01:01:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.698+0000 7faac3196640 1 -- 172.21.15.52:0/3963753802 wait complete. 2024-05-11T01:01:01.716 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:01.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.824+0000 7feecd56a640 1 Processor -- start 2024-05-11T01:01:01.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.824+0000 7feecd56a640 1 -- start start 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.825+0000 7feecd56a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec815c8b0 0x7feec806e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.825+0000 7feecd56a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feec80730d0 0x7feec806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.825+0000 7feecd56a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec806f360 con 0x7feec815b4c0 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.825+0000 7feecd56a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7feec806f5f0 con 0x7feec80730d0 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.825+0000 7feecd56a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7feec806f8d0 con 0x7feec815c8b0 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.825+0000 7feec77fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feec80730d0 0x7feec806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.825+0000 7feec77fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feec80730d0 0x7feec806ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60230/0 (socket says 172.21.15.52:60230) 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.825+0000 7feec77fe640 1 -- 172.21.15.52:0/3231890465 learned_addr learned my addr 172.21.15.52:0/3231890465 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.825+0000 7feec67fc640 1 --2- 172.21.15.52:0/3231890465 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec815c8b0 0x7feec806e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.826+0000 7feec77fe640 1 -- 172.21.15.52:0/3231890465 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec815c8b0 msgr2=0x7feec806e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.826+0000 7feec77fe640 1 --2- 172.21.15.52:0/3231890465 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec815c8b0 0x7feec806e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.826+0000 7feec77fe640 1 -- 172.21.15.52:0/3231890465 >> v1:172.21.15.119:6789/0 conn(0x7feec815b4c0 legacy=0x7feec80729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:01.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.826+0000 7feec77fe640 1 -- 172.21.15.52:0/3231890465 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec806fbb0 con 0x7feec80730d0 2024-05-11T01:01:01.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.826+0000 7feec77fe640 1 --2- 172.21.15.52:0/3231890465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec80730d0 0x7feec806ee20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feec000d150 tx=0x7feec002c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509413231099ca5e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:01.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feec5ffb640 1 -- 172.21.15.52:0/3231890465 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0009600 con 0x7feec80730d0 2024-05-11T01:01:01.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feec5ffb640 1 -- 172.21.15.52:0/3231890465 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec0037070 con 0x7feec80730d0 2024-05-11T01:01:01.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feec5ffb640 1 -- 172.21.15.52:0/3231890465 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00099c0 con 0x7feec80730d0 2024-05-11T01:01:01.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feecd56a640 1 -- 172.21.15.52:0/3231890465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec80730d0 msgr2=0x7feec806ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:01.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feecd56a640 1 --2- 172.21.15.52:0/3231890465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec80730d0 0x7feec806ee20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feec000d150 tx=0x7feec002c930 comp rx=0 tx=0).stop 2024-05-11T01:01:01.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feecd56a640 1 -- 172.21.15.52:0/3231890465 shutdown_connections 2024-05-11T01:01:01.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feecd56a640 1 --2- 172.21.15.52:0/3231890465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec80730d0 0x7feec806ee20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feecd56a640 1 --2- 172.21.15.52:0/3231890465 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec815c8b0 0x7feec806e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feecd56a640 1 -- 172.21.15.52:0/3231890465 >> 172.21.15.52:0/3231890465 conn(0x7feec80b2a00 msgr2=0x7feec80b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:01.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feecd56a640 1 -- 172.21.15.52:0/3231890465 shutdown_connections 2024-05-11T01:01:01.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.827+0000 7feecd56a640 1 -- 172.21.15.52:0/3231890465 wait complete. 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.829+0000 7feecd56a640 1 Processor -- start 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.829+0000 7feecd56a640 1 -- start start 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.829+0000 7feecd56a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec80730d0 0x7feec8151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.829+0000 7feecd56a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec815b4c0 0x7feec809dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.829+0000 7feecd56a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec815c8b0 0x7feec80b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.829+0000 7feecd56a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feec806e1d0 con 0x7feec815c8b0 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.829+0000 7feecd56a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feec806e050 con 0x7feec815b4c0 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.829+0000 7feecd56a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7feec806e350 con 0x7feec80730d0 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.829+0000 7feec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec815c8b0 0x7feec80b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec815c8b0 0x7feec80b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60244/0 (socket says 172.21.15.52:60244) 2024-05-11T01:01:01.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec77fe640 1 -- 172.21.15.52:0/3948287825 learned_addr learned my addr 172.21.15.52:0/3948287825 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec67fc640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec815b4c0 0x7feec809dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec6ffd640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec80730d0 0x7feec8151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec77fe640 1 -- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec80730d0 msgr2=0x7feec8151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec77fe640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec80730d0 0x7feec8151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec77fe640 1 -- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec815b4c0 msgr2=0x7feec809dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec77fe640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec815b4c0 0x7feec809dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec77fe640 1 -- 172.21.15.52:0/3948287825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec809e690 con 0x7feec815c8b0 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feec77fe640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec815c8b0 0x7feec80b5240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feec0034040 tx=0x7feec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feeaf7fe640 1 -- 172.21.15.52:0/3948287825 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0009dc0 con 0x7feec815c8b0 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feeaf7fe640 1 -- 172.21.15.52:0/3948287825 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec0047480 con 0x7feec815c8b0 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec81f11c0 con 0x7feec815c8b0 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feeaf7fe640 1 -- 172.21.15.52:0/3948287825 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0047870 con 0x7feec815c8b0 2024-05-11T01:01:01.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.830+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec81f1780 con 0x7feec815c8b0 2024-05-11T01:01:01.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.831+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec80b1680 con 0x7feec815c8b0 2024-05-11T01:01:01.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.832+0000 7feeaf7fe640 1 -- 172.21.15.52:0/3948287825 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feec0005d10 con 0x7feec815c8b0 2024-05-11T01:01:01.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.832+0000 7feeaf7fe640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7feeb40762d0 0x7feeb4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:01.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.832+0000 7feeaf7fe640 1 -- 172.21.15.52:0/3948287825 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7feec00baa30 con 0x7feec815c8b0 2024-05-11T01:01:01.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.832+0000 7feec6ffd640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7feeb40762d0 0x7feeb4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:01.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.833+0000 7feec6ffd640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7feeb40762d0 0x7feeb4078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feeb80017c0 tx=0x7feeb8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:01.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:01.837+0000 7feeaf7fe640 1 -- 172.21.15.52:0/3948287825 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feec80b1680 con 0x7feec815c8b0 2024-05-11T01:01:02.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.029+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feec8069780 con 0x7feec815c8b0 2024-05-11T01:01:02.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.030+0000 7feeaf7fe640 1 -- 172.21.15.52:0/3948287825 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7feec004c060 con 0x7feec815c8b0 2024-05-11T01:01:02.031 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:02.031 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":19,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:01.052717+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:00:59.086890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:00.008780+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:02.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7feeb40762d0 msgr2=0x7feeb4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7feeb40762d0 0x7feeb4078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feeb80017c0 tx=0x7feeb8009040 comp rx=0 tx=0).stop 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec815c8b0 msgr2=0x7feec80b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec815c8b0 0x7feec80b5240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feec0034040 tx=0x7feec0004a40 comp rx=0 tx=0).stop 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 shutdown_connections 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7feeb40762d0 0x7feeb4078790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec815c8b0 0x7feec80b5240 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec815b4c0 0x7feec809dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 --2- 172.21.15.52:0/3948287825 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feec80730d0 0x7feec8151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 >> 172.21.15.52:0/3948287825 conn(0x7feec80b2a00 msgr2=0x7feec8152790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:02.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 shutdown_connections 2024-05-11T01:01:02.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.032+0000 7feecd56a640 1 -- 172.21.15.52:0/3948287825 wait complete. 2024-05-11T01:01:02.044 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:02.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.148+0000 7fbee58a7640 1 Processor -- start 2024-05-11T01:01:02.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.149+0000 7fbee58a7640 1 -- start start 2024-05-11T01:01:02.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.149+0000 7fbee58a7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbee00b1260 0x7fbee00c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.149+0000 7fbee58a7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbee00d0d80 0x7fbee00d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.149+0000 7fbee58a7640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee00d02e0 con 0x7fbee0160730 2024-05-11T01:01:02.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.149+0000 7fbee58a7640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fbee00d08f0 con 0x7fbee00b1260 2024-05-11T01:01:02.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.149+0000 7fbee58a7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbee00c7d50 con 0x7fbee00d0d80 2024-05-11T01:01:02.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbedeffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbee00b1260 0x7fbee00c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbedeffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbee00b1260 0x7fbee00c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60256/0 (socket says 172.21.15.52:60256) 2024-05-11T01:01:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbedeffd640 1 -- 172.21.15.52:0/1686729437 learned_addr learned my addr 172.21.15.52:0/1686729437 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbede7fc640 1 --1- 172.21.15.52:0/1686729437 >> v1:172.21.15.52:6789/0 conn(0x7fbee0160730 0x7fbee006fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44610/0 (socket says 172.21.15.52:44610) 2024-05-11T01:01:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbedf7fe640 1 --2- 172.21.15.52:0/1686729437 >> v2:172.21.15.119:3300/0 conn(0x7fbee00d0d80 0x7fbee00d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbedeffd640 1 -- 172.21.15.52:0/1686729437 >> v2:172.21.15.119:3300/0 conn(0x7fbee00d0d80 msgr2=0x7fbee00d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbedeffd640 1 --2- 172.21.15.52:0/1686729437 >> v2:172.21.15.119:3300/0 conn(0x7fbee00d0d80 0x7fbee00d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbedeffd640 1 -- 172.21.15.52:0/1686729437 >> v1:172.21.15.52:6789/0 conn(0x7fbee0160730 legacy=0x7fbee006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbedeffd640 1 -- 172.21.15.52:0/1686729437 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee00c8030 con 0x7fbee00b1260 2024-05-11T01:01:02.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.150+0000 7fbeddffb640 1 -- 172.21.15.52:0/1686729437 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652661308 0 0) 0x7fbee00d02e0 con 0x7fbee0160730 2024-05-11T01:01:02.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbedeffd640 1 --2- 172.21.15.52:0/1686729437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00b1260 0x7fbee00c7690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbee00d0a70 tx=0x7fbed002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d4f7be4c592f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbeddffb640 1 -- 172.21.15.52:0/1686729437 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0037070 con 0x7fbee00b1260 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbeddffb640 1 -- 172.21.15.52:0/1686729437 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed0032440 con 0x7fbee00b1260 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbeddffb640 1 -- 172.21.15.52:0/1686729437 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0032810 con 0x7fbee00b1260 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbee58a7640 1 -- 172.21.15.52:0/1686729437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00b1260 msgr2=0x7fbee00c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbee58a7640 1 --2- 172.21.15.52:0/1686729437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00b1260 0x7fbee00c7690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbee00d0a70 tx=0x7fbed002a270 comp rx=0 tx=0).stop 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbee58a7640 1 -- 172.21.15.52:0/1686729437 shutdown_connections 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbee58a7640 1 --2- 172.21.15.52:0/1686729437 >> v2:172.21.15.119:3300/0 conn(0x7fbee00d0d80 0x7fbee00d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbee58a7640 1 --2- 172.21.15.52:0/1686729437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00b1260 0x7fbee00c7690 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbee58a7640 1 -- 172.21.15.52:0/1686729437 >> 172.21.15.52:0/1686729437 conn(0x7fbee00b3a10 msgr2=0x7fbee00b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbee58a7640 1 -- 172.21.15.52:0/1686729437 shutdown_connections 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.151+0000 7fbee58a7640 1 -- 172.21.15.52:0/1686729437 wait complete. 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.153+0000 7fbee58a7640 1 Processor -- start 2024-05-11T01:01:02.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.153+0000 7fbee58a7640 1 -- start start 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.153+0000 7fbee58a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee00b1260 0x7fbee0145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.153+0000 7fbee58a7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00d0d80 0x7fbee0154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.153+0000 7fbee58a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbee0160730 0x7fbee01f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.153+0000 7fbee58a7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0160b90 con 0x7fbee00d0d80 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.153+0000 7fbee58a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0070470 con 0x7fbee00b1260 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbee58a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbee00b1640 con 0x7fbee0160730 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbede7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00d0d80 0x7fbee0154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbede7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00d0d80 0x7fbee0154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60262/0 (socket says 172.21.15.52:60262) 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbede7fc640 1 -- 172.21.15.52:0/834455151 learned_addr learned my addr 172.21.15.52:0/834455151 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbedf7fe640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbee0160730 0x7fbee01f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbedeffd640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee00b1260 0x7fbee0145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbede7fc640 1 -- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbee0160730 msgr2=0x7fbee01f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbede7fc640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbee0160730 0x7fbee01f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbede7fc640 1 -- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee00b1260 msgr2=0x7fbee0145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbede7fc640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee00b1260 0x7fbee0145150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbede7fc640 1 -- 172.21.15.52:0/834455151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee01556f0 con 0x7fbee00d0d80 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbede7fc640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00d0d80 0x7fbee0154e90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c490 tx=0x7fbed400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbec7fff640 1 -- 172.21.15.52:0/834455151 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4018070 con 0x7fbee00d0d80 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbec7fff640 1 -- 172.21.15.52:0/834455151 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed4007500 con 0x7fbee00d0d80 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee0142690 con 0x7fbee00d0d80 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbec7fff640 1 -- 172.21.15.52:0/834455151 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4004a20 con 0x7fbee00d0d80 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.154+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee0142c50 con 0x7fbee00d0d80 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.155+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee0069780 con 0x7fbee00d0d80 2024-05-11T01:01:02.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.156+0000 7fbec7fff640 1 -- 172.21.15.52:0/834455151 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbed401b070 con 0x7fbee00d0d80 2024-05-11T01:01:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.156+0000 7fbec7fff640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbecc0762d0 0x7fbecc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.156+0000 7fbec7fff640 1 -- 172.21.15.52:0/834455151 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fbed40968b0 con 0x7fbee00d0d80 2024-05-11T01:01:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.157+0000 7fbedeffd640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbecc0762d0 0x7fbecc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.157+0000 7fbedeffd640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbecc0762d0 0x7fbecc078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbed0034000 tx=0x7fbed002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:02.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.161+0000 7fbec7fff640 1 -- 172.21.15.52:0/834455151 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbed405fbc0 con 0x7fbee00d0d80 2024-05-11T01:01:02.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.351+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbee006f630 con 0x7fbee00d0d80 2024-05-11T01:01:02.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.352+0000 7fbec7fff640 1 -- 172.21.15.52:0/834455151 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7fbed4063ec0 con 0x7fbee00d0d80 2024-05-11T01:01:02.353 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:02.353 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":19,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:01.052717+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:00:59.086890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:00.008780+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:02.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbecc0762d0 msgr2=0x7fbecc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbecc0762d0 0x7fbecc078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbed0034000 tx=0x7fbed002e000 comp rx=0 tx=0).stop 2024-05-11T01:01:02.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00d0d80 msgr2=0x7fbee0154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00d0d80 0x7fbee0154e90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c490 tx=0x7fbed400c950 comp rx=0 tx=0).stop 2024-05-11T01:01:02.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 shutdown_connections 2024-05-11T01:01:02.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbecc0762d0 0x7fbecc078790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbee0160730 0x7fbee01f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee00d0d80 0x7fbee0154e90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 --2- 172.21.15.52:0/834455151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbee00b1260 0x7fbee0145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.354+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 >> 172.21.15.52:0/834455151 conn(0x7fbee00b3a10 msgr2=0x7fbee01590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:02.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.355+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 shutdown_connections 2024-05-11T01:01:02.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.355+0000 7fbee58a7640 1 -- 172.21.15.52:0/834455151 wait complete. 2024-05-11T01:01:02.366 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T01:01:02.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.473+0000 7f0759746640 1 Processor -- start 2024-05-11T01:01:02.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f0759746640 1 -- start start 2024-05-11T01:01:02.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f0759746640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07540cd7f0 0x7f0754073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f0759746640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0754073790 0x7f075406f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f0759746640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07540cde60 con 0x7f075415c430 2024-05-11T01:01:02.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f0759746640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f07540ce140 con 0x7f07540cd7f0 2024-05-11T01:01:02.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f0759746640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f07540ce420 con 0x7f0754073790 2024-05-11T01:01:02.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f07527fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07540cd7f0 0x7f0754073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f07527fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f07540cd7f0 0x7f0754073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60266/0 (socket says 172.21.15.52:60266) 2024-05-11T01:01:02.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f07527fc640 1 -- 172.21.15.52:0/3445419656 learned_addr learned my addr 172.21.15.52:0/3445419656 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:02.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.474+0000 7f0752ffd640 1 --1- 172.21.15.52:0/3445419656 >> v1:172.21.15.52:6789/0 conn(0x7f075415c430 0x7f07540d0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44614/0 (socket says 172.21.15.52:44614) 2024-05-11T01:01:02.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.475+0000 7f07537fe640 1 --2- 172.21.15.52:0/3445419656 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0754073790 0x7f075406f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.475+0000 7f07527fc640 1 -- 172.21.15.52:0/3445419656 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0754073790 msgr2=0x7f075406f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:02.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.475+0000 7f07527fc640 1 --2- 172.21.15.52:0/3445419656 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0754073790 0x7f075406f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.475+0000 7f07527fc640 1 -- 172.21.15.52:0/3445419656 >> v1:172.21.15.52:6789/0 conn(0x7f075415c430 legacy=0x7f07540d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.475+0000 7f07527fc640 1 -- 172.21.15.52:0/3445419656 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075406f170 con 0x7f07540cd7f0 2024-05-11T01:01:02.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.475+0000 7f0751ffb640 1 -- 172.21.15.52:0/3445419656 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422421822 0 0) 0x7f07540cde60 con 0x7f075415c430 2024-05-11T01:01:02.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.475+0000 7f07527fc640 1 --2- 172.21.15.52:0/3445419656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 0x7f0754073250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f073c009870 tx=0x7f073c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dc2db33f0a82fc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:02.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0751ffb640 1 -- 172.21.15.52:0/3445419656 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c037070 con 0x7f07540cd7f0 2024-05-11T01:01:02.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0751ffb640 1 -- 172.21.15.52:0/3445419656 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073c032440 con 0x7f07540cd7f0 2024-05-11T01:01:02.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0751ffb640 1 -- 172.21.15.52:0/3445419656 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c032810 con 0x7f07540cd7f0 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0759746640 1 -- 172.21.15.52:0/3445419656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 msgr2=0x7f0754073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0759746640 1 --2- 172.21.15.52:0/3445419656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 0x7f0754073250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f073c009870 tx=0x7f073c02a270 comp rx=0 tx=0).stop 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0759746640 1 -- 172.21.15.52:0/3445419656 shutdown_connections 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0759746640 1 --2- 172.21.15.52:0/3445419656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 0x7f0754073250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0759746640 1 --2- 172.21.15.52:0/3445419656 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0754073790 0x7f075406f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0759746640 1 -- 172.21.15.52:0/3445419656 >> 172.21.15.52:0/3445419656 conn(0x7f07540d1ef0 msgr2=0x7f07540d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0759746640 1 -- 172.21.15.52:0/3445419656 shutdown_connections 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.476+0000 7f0759746640 1 -- 172.21.15.52:0/3445419656 wait complete. 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.477+0000 7f0759746640 1 Processor -- start 2024-05-11T01:01:02.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f0759746640 1 -- start start 2024-05-11T01:01:02.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f0759746640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0754073790 0x7f0754164e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f0759746640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 0x7f075415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f0759746640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f075415c430 0x7f07540b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f0759746640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f075406ff60 con 0x7f07540cd7f0 2024-05-11T01:01:02.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f0759746640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f075406fde0 con 0x7f0754073790 2024-05-11T01:01:02.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f0759746640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f07540700e0 con 0x7f075415c430 2024-05-11T01:01:02.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f07527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 0x7f075415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f07527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 0x7f075415f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60280/0 (socket says 172.21.15.52:60280) 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.478+0000 7f07527fc640 1 -- 172.21.15.52:0/3996891939 learned_addr learned my addr 172.21.15.52:0/3996891939 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f07537fe640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f075415c430 0x7f07540b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f07527fc640 1 -- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f075415c430 msgr2=0x7f07540b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f07527fc640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f075415c430 0x7f07540b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f07527fc640 1 -- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0754073790 msgr2=0x7f0754164e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f07527fc640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0754073790 0x7f0754164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f07527fc640 1 -- 172.21.15.52:0/3996891939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07541667a0 con 0x7f07540cd7f0 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f07527fc640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 0x7f075415f040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f073c009870 tx=0x7f073c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f074b7fe640 1 -- 172.21.15.52:0/3996891939 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c041070 con 0x7f07540cd7f0 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f074b7fe640 1 -- 172.21.15.52:0/3996891939 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073c03c890 con 0x7f07540cd7f0 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07541640a0 con 0x7f07540cd7f0 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f074b7fe640 1 -- 172.21.15.52:0/3996891939 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c037040 con 0x7f07540cd7f0 2024-05-11T01:01:02.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.479+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0754164660 con 0x7f07540cd7f0 2024-05-11T01:01:02.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.480+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0754069c00 con 0x7f07540cd7f0 2024-05-11T01:01:02.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.481+0000 7f074b7fe640 1 -- 172.21.15.52:0/3996891939 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f073c044070 con 0x7f07540cd7f0 2024-05-11T01:01:02.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.481+0000 7f074b7fe640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07340762d0 0x7f0734078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:02.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.481+0000 7f074b7fe640 1 -- 172.21.15.52:0/3996891939 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f073c0b8080 con 0x7f07540cd7f0 2024-05-11T01:01:02.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.481+0000 7f0752ffd640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07340762d0 0x7f0734078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:02.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.482+0000 7f0752ffd640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07340762d0 0x7f0734078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f074c007fd0 tx=0x7f074c007ce0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:02.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.485+0000 7f074b7fe640 1 -- 172.21.15.52:0/3996891939 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f073c0819e0 con 0x7f07540cd7f0 2024-05-11T01:01:02.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:02.718+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f07540b0180 con 0x7f07540cd7f0 2024-05-11T01:01:03.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.693+0000 7f074b7fe640 1 -- 172.21.15.52:0/3996891939 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f073c0812e0 con 0x7f07540cd7f0 2024-05-11T01:01:03.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07340762d0 msgr2=0x7f0734078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:03.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07340762d0 0x7f0734078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f074c007fd0 tx=0x7f074c007ce0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 msgr2=0x7f075415f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:03.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 0x7f075415f040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f073c009870 tx=0x7f073c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 shutdown_connections 2024-05-11T01:01:03.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07340762d0 0x7f0734078790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f075415c430 0x7f07540b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07540cd7f0 0x7f075415f040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 --2- 172.21.15.52:0/3996891939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0754073790 0x7f0754164e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 >> 172.21.15.52:0/3996891939 conn(0x7f07540d1ef0 msgr2=0x7f07540b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:03.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 shutdown_connections 2024-05-11T01:01:03.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.696+0000 7f0759746640 1 -- 172.21.15.52:0/3996891939 wait complete. 2024-05-11T01:01:03.710 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T01:01:03.710 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T01:01:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.823+0000 7ffabbf5f640 1 Processor -- start 2024-05-11T01:01:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.823+0000 7ffabbf5f640 1 -- start start 2024-05-11T01:01:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffabbf5f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffab40d2b40 0x7ffab406d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffabbf5f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 0x7ffab40b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffabbf5f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab406e140 con 0x7ffab40b4320 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffabbf5f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffab406dd80 con 0x7ffab40d2b40 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffabbf5f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffab406e420 con 0x7ffab409c130 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab9cfb640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7ffab40b4320 0x7ffab409ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44628/0 (socket says 172.21.15.52:44628) 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab9cfb640 1 -- 172.21.15.52:0/2382766043 learned_addr learned my addr 172.21.15.52:0/2382766043 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab94fa640 1 --2- 172.21.15.52:0/2382766043 >> v2:172.21.15.119:3300/0 conn(0x7ffab40d2b40 0x7ffab406d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffaba4fc640 1 --2- 172.21.15.52:0/2382766043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 0x7ffab40b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab8cf9640 1 -- 172.21.15.52:0/2382766043 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3786716518 0 0) 0x7ffab406e140 con 0x7ffab40b4320 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab8cf9640 1 -- 172.21.15.52:0/2382766043 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaa0003520 con 0x7ffab40b4320 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab94fa640 1 -- 172.21.15.52:0/2382766043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 msgr2=0x7ffab40b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab8cf9640 1 -- 172.21.15.52:0/2382766043 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 71348185 0 0) 0x7ffaa0003520 con 0x7ffab40b4320 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab94fa640 1 --2- 172.21.15.52:0/2382766043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 0x7ffab40b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab94fa640 1 -- 172.21.15.52:0/2382766043 >> v1:172.21.15.52:6789/0 conn(0x7ffab40b4320 legacy=0x7ffab409ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffab94fa640 1 -- 172.21.15.52:0/2382766043 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab40d1e60 con 0x7ffab40d2b40 2024-05-11T01:01:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.824+0000 7ffaba4fc640 1 --2- 172.21.15.52:0/2382766043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 0x7ffab40b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.825+0000 7ffab94fa640 1 --2- 172.21.15.52:0/2382766043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 0x7ffab406d7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40097a0 tx=0x7ffaa402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab635f2d6494b12 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.825+0000 7ffab8cf9640 1 -- 172.21.15.52:0/2382766043 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4037070 con 0x7ffab40d2b40 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.825+0000 7ffab8cf9640 1 -- 172.21.15.52:0/2382766043 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa402ab10 con 0x7ffab40d2b40 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.825+0000 7ffab8cf9640 1 -- 172.21.15.52:0/2382766043 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab00016f0 con 0x7ffab40d2b40 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.826+0000 7ffabbf5f640 1 -- 172.21.15.52:0/2382766043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 msgr2=0x7ffab406d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.826+0000 7ffabbf5f640 1 --2- 172.21.15.52:0/2382766043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 0x7ffab406d7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40097a0 tx=0x7ffaa402a080 comp rx=0 tx=0).stop 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.826+0000 7ffabbf5f640 1 -- 172.21.15.52:0/2382766043 shutdown_connections 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.826+0000 7ffabbf5f640 1 --2- 172.21.15.52:0/2382766043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 0x7ffab40b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.826+0000 7ffabbf5f640 1 --2- 172.21.15.52:0/2382766043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 0x7ffab406d7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.826+0000 7ffabbf5f640 1 -- 172.21.15.52:0/2382766043 >> 172.21.15.52:0/2382766043 conn(0x7ffab40b4a60 msgr2=0x7ffab40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.826+0000 7ffabbf5f640 1 -- 172.21.15.52:0/2382766043 shutdown_connections 2024-05-11T01:01:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.826+0000 7ffabbf5f640 1 -- 172.21.15.52:0/2382766043 wait complete. 2024-05-11T01:01:03.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.827+0000 7ffabbf5f640 1 Processor -- start 2024-05-11T01:01:03.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.827+0000 7ffabbf5f640 1 -- start start 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffabbf5f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 0x7ffab415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffabbf5f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffab40b4320 0x7ffab4162520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffabbf5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 0x7ffab40cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffabbf5f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffab40b5980 con 0x7ffab40b4320 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffabbf5f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffab40b5800 con 0x7ffab40d2b40 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffabbf5f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffab40d2380 con 0x7ffab409c130 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffab94fa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffab40b4320 0x7ffab4162520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffab94fa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffab40b4320 0x7ffab4162520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60294/0 (socket says 172.21.15.52:60294) 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffab94fa640 1 -- 172.21.15.52:0/697375684 learned_addr learned my addr 172.21.15.52:0/697375684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffab9cfb640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 0x7ffab415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffaba4fc640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 0x7ffab40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffab94fa640 1 -- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 msgr2=0x7ffab415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffab94fa640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 0x7ffab415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffab94fa640 1 -- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 msgr2=0x7ffab40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffab94fa640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 0x7ffab40cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffab94fa640 1 -- 172.21.15.52:0/697375684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab40c6b70 con 0x7ffab40b4320 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.828+0000 7ffaba4fc640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 0x7ffab40cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.829+0000 7ffab94fa640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffab40b4320 0x7ffab4162520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40098c0 tx=0x7ffaa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.829+0000 7ffaaa7fc640 1 -- 172.21.15.52:0/697375684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4042020 con 0x7ffab40b4320 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.829+0000 7ffaaa7fc640 1 -- 172.21.15.52:0/697375684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa403d670 con 0x7ffab40b4320 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.829+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab40c6e10 con 0x7ffab40b4320 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.829+0000 7ffaaa7fc640 1 -- 172.21.15.52:0/697375684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4037040 con 0x7ffab40b4320 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.829+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab40c73d0 con 0x7ffab40b4320 2024-05-11T01:01:03.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.830+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab40695d0 con 0x7ffab40b4320 2024-05-11T01:01:03.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.830+0000 7ffaaa7fc640 1 -- 172.21.15.52:0/697375684 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffaa4045020 con 0x7ffab40b4320 2024-05-11T01:01:03.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.831+0000 7ffaaa7fc640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa980762d0 0x7ffa98078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:03.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.831+0000 7ffaaa7fc640 1 -- 172.21.15.52:0/697375684 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffaa40b7ed0 con 0x7ffab40b4320 2024-05-11T01:01:03.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.831+0000 7ffab9cfb640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa980762d0 0x7ffa98078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:03.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.832+0000 7ffab9cfb640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa980762d0 0x7ffa98078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffab0005880 tx=0x7ffab0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:03.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:03.835+0000 7ffaaa7fc640 1 -- 172.21.15.52:0/697375684 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffaa404c030 con 0x7ffab40b4320 2024-05-11T01:01:04.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.064+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ffab40b8ef0 con 0x7ffab40b4320 2024-05-11T01:01:04.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.694+0000 7ffaaa7fc640 1 -- 172.21.15.52:0/697375684 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7ffaa4081830 con 0x7ffab40b4320 2024-05-11T01:01:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa980762d0 msgr2=0x7ffa98078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa980762d0 0x7ffa98078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffab0005880 tx=0x7ffab0009040 comp rx=0 tx=0).stop 2024-05-11T01:01:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffab40b4320 msgr2=0x7ffab4162520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffab40b4320 0x7ffab4162520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40098c0 tx=0x7ffaa40046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 shutdown_connections 2024-05-11T01:01:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa980762d0 0x7ffa98078790 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab40d2b40 0x7ffab40cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffab40b4320 0x7ffab4162520 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 --2- 172.21.15.52:0/697375684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffab409c130 0x7ffab415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 >> 172.21.15.52:0/697375684 conn(0x7ffab40b4a60 msgr2=0x7ffab4071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 shutdown_connections 2024-05-11T01:01:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.697+0000 7ffabbf5f640 1 -- 172.21.15.52:0/697375684 wait complete. 2024-05-11T01:01:04.710 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T01:01:04.710 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:04.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.819+0000 7f529baf1640 1 Processor -- start 2024-05-11T01:01:04.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.820+0000 7f529baf1640 1 -- start start 2024-05-11T01:01:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529baf1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f52940b5dd0 0x7f52940d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529baf1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5294163250 con 0x7f529415f430 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529baf1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5294163530 con 0x7f5294160480 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529baf1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5294163810 con 0x7f52940b5dd0 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529988d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f529415f430 0x7f529415fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44638/0 (socket says 172.21.15.52:44638) 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529988d640 1 -- 172.21.15.52:0/3887329354 learned_addr learned my addr 172.21.15.52:0/3887329354 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529a08e640 1 --2- 172.21.15.52:0/3887329354 >> v2:172.21.15.119:3300/0 conn(0x7f52940b5dd0 0x7f52940d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529888b640 1 -- 172.21.15.52:0/3887329354 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606916971 0 0) 0x7f5294163250 con 0x7f529415f430 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529888b640 1 -- 172.21.15.52:0/3887329354 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5280003520 con 0x7f529415f430 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529888b640 1 -- 172.21.15.52:0/3887329354 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1488305502 0 0) 0x7f5294163530 con 0x7f5294160480 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529888b640 1 -- 172.21.15.52:0/3887329354 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5294163250 con 0x7f5294160480 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529a08e640 1 -- 172.21.15.52:0/3887329354 >> v1:172.21.15.119:6789/0 conn(0x7f5294160480 legacy=0x7f52940d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529a08e640 1 -- 172.21.15.52:0/3887329354 >> v1:172.21.15.52:6789/0 conn(0x7f529415f430 legacy=0x7f529415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529888b640 1 -- 172.21.15.52:0/3887329354 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3196714385 0 0) 0x7f5280003520 con 0x7f529415f430 2024-05-11T01:01:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.821+0000 7f529a08e640 1 -- 172.21.15.52:0/3887329354 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5294163af0 con 0x7f52940b5dd0 2024-05-11T01:01:04.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.822+0000 7f529a08e640 1 --2- 172.21.15.52:0/3887329354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52940b5dd0 0x7f52940d3490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f528c009d90 tx=0x7f528c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=566586af6d461c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.822+0000 7f529888b640 1 -- 172.21.15.52:0/3887329354 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52900016f0 con 0x7f52940b5dd0 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.822+0000 7f529888b640 1 -- 172.21.15.52:0/3887329354 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528c007540 con 0x7f52940b5dd0 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.822+0000 7f529888b640 1 -- 172.21.15.52:0/3887329354 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c007910 con 0x7f52940b5dd0 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.823+0000 7f529baf1640 1 -- 172.21.15.52:0/3887329354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52940b5dd0 msgr2=0x7f52940d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.823+0000 7f529baf1640 1 --2- 172.21.15.52:0/3887329354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52940b5dd0 0x7f52940d3490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f528c009d90 tx=0x7f528c02a5f0 comp rx=0 tx=0).stop 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.823+0000 7f529baf1640 1 -- 172.21.15.52:0/3887329354 shutdown_connections 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.823+0000 7f529baf1640 1 --2- 172.21.15.52:0/3887329354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52940b5dd0 0x7f52940d3490 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.823+0000 7f529baf1640 1 -- 172.21.15.52:0/3887329354 >> 172.21.15.52:0/3887329354 conn(0x7f529408eb70 msgr2=0x7f529409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.823+0000 7f529baf1640 1 -- 172.21.15.52:0/3887329354 shutdown_connections 2024-05-11T01:01:04.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.823+0000 7f529baf1640 1 -- 172.21.15.52:0/3887329354 wait complete. 2024-05-11T01:01:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.824+0000 7f529baf1640 1 Processor -- start 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.824+0000 7f529baf1640 1 -- start start 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529baf1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52940b5dd0 0x7f529415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529baf1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f529415f430 0x7f52940ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529baf1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5294160480 0x7f52940d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529baf1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5294163030 con 0x7f52940b5dd0 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529baf1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5294162eb0 con 0x7f529415f430 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529baf1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f52941608f0 con 0x7f5294160480 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529988d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52940b5dd0 0x7f529415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529988d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52940b5dd0 0x7f529415a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60306/0 (socket says 172.21.15.52:60306) 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529988d640 1 -- 172.21.15.52:0/2115183549 learned_addr learned my addr 172.21.15.52:0/2115183549 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529a08e640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5294160480 0x7f52940d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529908c640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f529415f430 0x7f52940ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529988d640 1 -- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5294160480 msgr2=0x7f52940d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529988d640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5294160480 0x7f52940d2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529988d640 1 -- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f529415f430 msgr2=0x7f52940ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529988d640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f529415f430 0x7f52940ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529988d640 1 -- 172.21.15.52:0/2115183549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529415c030 con 0x7f52940b5dd0 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.825+0000 7f529988d640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52940b5dd0 0x7f529415a5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f529000c590 tx=0x7f529000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.826+0000 7f528a7fc640 1 -- 172.21.15.52:0/2115183549 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290017070 con 0x7f52940b5dd0 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.826+0000 7f528a7fc640 1 -- 172.21.15.52:0/2115183549 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5290005ce0 con 0x7f52940b5dd0 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.826+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529415c2d0 con 0x7f52940b5dd0 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.826+0000 7f528a7fc640 1 -- 172.21.15.52:0/2115183549 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290007290 con 0x7f52940b5dd0 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.826+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529415c890 con 0x7f52940b5dd0 2024-05-11T01:01:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.827+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5294069780 con 0x7f52940b5dd0 2024-05-11T01:01:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.827+0000 7f528a7fc640 1 -- 172.21.15.52:0/2115183549 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f529001a070 con 0x7f52940b5dd0 2024-05-11T01:01:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.828+0000 7f528a7fc640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f52780762d0 0x7f5278078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.828+0000 7f528a7fc640 1 -- 172.21.15.52:0/2115183549 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5290096af0 con 0x7f52940b5dd0 2024-05-11T01:01:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.828+0000 7f529908c640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f52780762d0 0x7f5278078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.829+0000 7f529908c640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f52780762d0 0x7f5278078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f52940a3040 tx=0x7f5284009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:04.832+0000 7f528a7fc640 1 -- 172.21.15.52:0/2115183549 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5290060450 con 0x7f52940b5dd0 2024-05-11T01:01:05.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.061+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52940b4ba0 con 0x7f52940b5dd0 2024-05-11T01:01:05.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.061+0000 7f528a7fc640 1 -- 172.21.15.52:0/2115183549 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f529005fd50 con 0x7f52940b5dd0 2024-05-11T01:01:05.062 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:05.062 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2245862597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2245862597},{"type":"v1","addr":"172.21.15.119:6837","nonce":2245862597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4286578298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4286578298},{"type":"v1","addr":"172.21.15.52:6835","nonce":4286578298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:01.052672+0000","modified":"2024-05-11T01:01:04.065799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/1484114523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1484114523},{"type":"v1","addr":"172.21.15.119:6836","nonce":1484114523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-11T01:01:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f52780762d0 msgr2=0x7f5278078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f52780762d0 0x7f5278078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f52940a3040 tx=0x7f5284009040 comp rx=0 tx=0).stop 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52940b5dd0 msgr2=0x7f529415a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52940b5dd0 0x7f529415a5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f529000c590 tx=0x7f529000ca50 comp rx=0 tx=0).stop 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 shutdown_connections 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f52780762d0 0x7f5278078790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5294160480 0x7f52940d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f529415f430 0x7f52940ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 --2- 172.21.15.52:0/2115183549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52940b5dd0 0x7f529415a5c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 >> 172.21.15.52:0/2115183549 conn(0x7f529408eb70 msgr2=0x7f529415d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:05.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 shutdown_connections 2024-05-11T01:01:05.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.063+0000 7f529baf1640 1 -- 172.21.15.52:0/2115183549 wait complete. 2024-05-11T01:01:05.075 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.179+0000 7ff05af14640 1 Processor -- start 2024-05-11T01:01:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.179+0000 7ff05af14640 1 -- start start 2024-05-11T01:01:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.179+0000 7ff05af14640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff05415f430 0x7ff05415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.179+0000 7ff05af14640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0541602b0 0x7ff0540d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.179+0000 7ff05af14640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff054163250 con 0x7ff054160720 2024-05-11T01:01:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.179+0000 7ff05af14640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff054163530 con 0x7ff05415f430 2024-05-11T01:01:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.179+0000 7ff05af14640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff054163810 con 0x7ff0541602b0 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.180+0000 7ff058cb0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff05415f430 0x7ff05415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.180+0000 7ff058cb0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff05415f430 0x7ff05415fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60312/0 (socket says 172.21.15.52:60312) 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.180+0000 7ff058cb0640 1 -- 172.21.15.52:0/2038226652 learned_addr learned my addr 172.21.15.52:0/2038226652 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.180+0000 7ff053fff640 1 --2- 172.21.15.52:0/2038226652 >> v2:172.21.15.119:3300/0 conn(0x7ff0541602b0 0x7ff0540d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.180+0000 7ff058cb0640 1 -- 172.21.15.52:0/2038226652 >> v2:172.21.15.119:3300/0 conn(0x7ff0541602b0 msgr2=0x7ff0540d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.180+0000 7ff058cb0640 1 --2- 172.21.15.52:0/2038226652 >> v2:172.21.15.119:3300/0 conn(0x7ff0541602b0 0x7ff0540d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.180+0000 7ff058cb0640 1 -- 172.21.15.52:0/2038226652 >> v1:172.21.15.119:6789/0 conn(0x7ff054160720 legacy=0x7ff0540d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.180+0000 7ff058cb0640 1 -- 172.21.15.52:0/2038226652 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff054163af0 con 0x7ff05415f430 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.180+0000 7ff053fff640 1 --2- 172.21.15.52:0/2038226652 >> v2:172.21.15.119:3300/0 conn(0x7ff0541602b0 0x7ff0540d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff058cb0640 1 --2- 172.21.15.52:0/2038226652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 0x7ff05415fd70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0540b4840 tx=0x7ff04802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b9fc2bc7ce252e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:05.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff0537fe640 1 -- 172.21.15.52:0/2038226652 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048037070 con 0x7ff05415f430 2024-05-11T01:01:05.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff0537fe640 1 -- 172.21.15.52:0/2038226652 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff048032440 con 0x7ff05415f430 2024-05-11T01:01:05.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff0537fe640 1 -- 172.21.15.52:0/2038226652 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048032810 con 0x7ff05415f430 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff05af14640 1 -- 172.21.15.52:0/2038226652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 msgr2=0x7ff05415fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff05af14640 1 --2- 172.21.15.52:0/2038226652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 0x7ff05415fd70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0540b4840 tx=0x7ff04802a140 comp rx=0 tx=0).stop 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff05af14640 1 -- 172.21.15.52:0/2038226652 shutdown_connections 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff05af14640 1 --2- 172.21.15.52:0/2038226652 >> v2:172.21.15.119:3300/0 conn(0x7ff0541602b0 0x7ff0540d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff05af14640 1 --2- 172.21.15.52:0/2038226652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 0x7ff05415fd70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff05af14640 1 -- 172.21.15.52:0/2038226652 >> 172.21.15.52:0/2038226652 conn(0x7ff05408eb70 msgr2=0x7ff05409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff05af14640 1 -- 172.21.15.52:0/2038226652 shutdown_connections 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.181+0000 7ff05af14640 1 -- 172.21.15.52:0/2038226652 wait complete. 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.182+0000 7ff05af14640 1 Processor -- start 2024-05-11T01:01:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.183+0000 7ff05af14640 1 -- start start 2024-05-11T01:01:05.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.183+0000 7ff05af14640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 0x7ff0541ea310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.183+0000 7ff05af14640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0541602b0 0x7ff05415db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.183+0000 7ff05af14640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff054160720 0x7ff0540cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.183+0000 7ff05af14640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff054163030 con 0x7ff05415f430 2024-05-11T01:01:05.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.183+0000 7ff05af14640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff054162eb0 con 0x7ff054160720 2024-05-11T01:01:05.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.183+0000 7ff05af14640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff0540d27a0 con 0x7ff0541602b0 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.183+0000 7ff058cb0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 0x7ff0541ea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff058cb0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 0x7ff0541ea310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60318/0 (socket says 172.21.15.52:60318) 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff058cb0640 1 -- 172.21.15.52:0/3392494212 learned_addr learned my addr 172.21.15.52:0/3392494212 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff058cb0640 1 -- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0541602b0 msgr2=0x7ff05415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff0594b1640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff054160720 0x7ff0540cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff053fff640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0541602b0 0x7ff05415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff058cb0640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0541602b0 0x7ff05415db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff058cb0640 1 -- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff054160720 msgr2=0x7ff0540cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff058cb0640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff054160720 0x7ff0540cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff058cb0640 1 -- 172.21.15.52:0/3392494212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05415e040 con 0x7ff05415f430 2024-05-11T01:01:05.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff058cb0640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 0x7ff0541ea310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff048000c00 tx=0x7ff0480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:05.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff0517fa640 1 -- 172.21.15.52:0/3392494212 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048037040 con 0x7ff05415f430 2024-05-11T01:01:05.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff0517fa640 1 -- 172.21.15.52:0/3392494212 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff048042070 con 0x7ff05415f430 2024-05-11T01:01:05.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0541e9270 con 0x7ff05415f430 2024-05-11T01:01:05.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff0517fa640 1 -- 172.21.15.52:0/3392494212 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048030bb0 con 0x7ff05415f430 2024-05-11T01:01:05.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.184+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0541e9830 con 0x7ff05415f430 2024-05-11T01:01:05.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.185+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff054069780 con 0x7ff05415f430 2024-05-11T01:01:05.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.186+0000 7ff0517fa640 1 -- 172.21.15.52:0/3392494212 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff048047460 con 0x7ff05415f430 2024-05-11T01:01:05.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.187+0000 7ff0517fa640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff0380762d0 0x7ff038078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.187+0000 7ff0517fa640 1 -- 172.21.15.52:0/3392494212 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff0480b82a0 con 0x7ff05415f430 2024-05-11T01:01:05.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.187+0000 7ff053fff640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff0380762d0 0x7ff038078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.187+0000 7ff053fff640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff0380762d0 0x7ff038078790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff054076190 tx=0x7ff040008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:05.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.191+0000 7ff0517fa640 1 -- 172.21.15.52:0/3392494212 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff048081b60 con 0x7ff05415f430 2024-05-11T01:01:05.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.383+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0541e9f30 con 0x7ff05415f430 2024-05-11T01:01:05.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.384+0000 7ff0517fa640 1 -- 172.21.15.52:0/3392494212 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7ff048081460 con 0x7ff05415f430 2024-05-11T01:01:05.385 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:05.385 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":20,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:02.673511+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:00:59.086890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:00.008780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.386+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff0380762d0 msgr2=0x7ff038078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.386+0000 7ff05af14640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff0380762d0 0x7ff038078790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff054076190 tx=0x7ff040008040 comp rx=0 tx=0).stop 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.386+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 msgr2=0x7ff0541ea310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.386+0000 7ff05af14640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 0x7ff0541ea310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff048000c00 tx=0x7ff0480046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.386+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 shutdown_connections 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.386+0000 7ff05af14640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff0380762d0 0x7ff038078790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.386+0000 7ff05af14640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff054160720 0x7ff0540cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.386+0000 7ff05af14640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff0541602b0 0x7ff05415db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.386+0000 7ff05af14640 1 --2- 172.21.15.52:0/3392494212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff05415f430 0x7ff0541ea310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.387+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 >> 172.21.15.52:0/3392494212 conn(0x7ff05408eb70 msgr2=0x7ff0540b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.387+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 shutdown_connections 2024-05-11T01:01:05.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.387+0000 7ff05af14640 1 -- 172.21.15.52:0/3392494212 wait complete. 2024-05-11T01:01:05.398 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T01:01:05.398 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:01:05.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.501+0000 7fa3f1590640 1 Processor -- start 2024-05-11T01:01:05.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.502+0000 7fa3f1590640 1 -- start start 2024-05-11T01:01:05.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.502+0000 7fa3f1590640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa3ec0b64c0 0x7fa3ec073940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.502+0000 7fa3f1590640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa3ec0cfd80 0x7fa3ec0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.502+0000 7fa3f1590640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec0d14a0 con 0x7fa3ec142a30 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.502+0000 7fa3f1590640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ec142e10 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.502+0000 7fa3f1590640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ec0d1780 con 0x7fa3ec0b64c0 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.502+0000 7fa3eb7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa3ec0cfd80 0x7fa3ec0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.503+0000 7fa3eb7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa3ec0cfd80 0x7fa3ec0d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60328/0 (socket says 172.21.15.52:60328) 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.503+0000 7fa3eb7fe640 1 -- 172.21.15.52:0/2030297095 learned_addr learned my addr 172.21.15.52:0/2030297095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.503+0000 7fa3e27fc640 1 --2- 172.21.15.52:0/2030297095 >> v2:172.21.15.119:3300/0 conn(0x7fa3ec0b64c0 0x7fa3ec073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.503+0000 7fa3eb7fe640 1 -- 172.21.15.52:0/2030297095 >> v2:172.21.15.119:3300/0 conn(0x7fa3ec0b64c0 msgr2=0x7fa3ec073940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.503+0000 7fa3eb7fe640 1 --2- 172.21.15.52:0/2030297095 >> v2:172.21.15.119:3300/0 conn(0x7fa3ec0b64c0 0x7fa3ec073940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.503+0000 7fa3eb7fe640 1 -- 172.21.15.52:0/2030297095 >> v1:172.21.15.119:6789/0 conn(0x7fa3ec142a30 legacy=0x7fa3ec073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.503+0000 7fa3eb7fe640 1 -- 172.21.15.52:0/2030297095 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec0d1a60 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3eb7fe640 1 --2- 172.21.15.52:0/2030297095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 0x7fa3ec0d01b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec06a5b0 tx=0x7fa3e402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7eb47fb8fdc51a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:05.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3ea7fc640 1 -- 172.21.15.52:0/2030297095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4036070 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3ea7fc640 1 -- 172.21.15.52:0/2030297095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e4031600 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3ea7fc640 1 -- 172.21.15.52:0/2030297095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40319d0 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3f1590640 1 -- 172.21.15.52:0/2030297095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 msgr2=0x7fa3ec0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3f1590640 1 --2- 172.21.15.52:0/2030297095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 0x7fa3ec0d01b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec06a5b0 tx=0x7fa3e402a730 comp rx=0 tx=0).stop 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3f1590640 1 -- 172.21.15.52:0/2030297095 shutdown_connections 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3f1590640 1 --2- 172.21.15.52:0/2030297095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 0x7fa3ec0d01b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3f1590640 1 --2- 172.21.15.52:0/2030297095 >> v2:172.21.15.119:3300/0 conn(0x7fa3ec0b64c0 0x7fa3ec073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3f1590640 1 -- 172.21.15.52:0/2030297095 >> 172.21.15.52:0/2030297095 conn(0x7fa3ec0b3af0 msgr2=0x7fa3ec0b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3f1590640 1 -- 172.21.15.52:0/2030297095 shutdown_connections 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.504+0000 7fa3f1590640 1 -- 172.21.15.52:0/2030297095 wait complete. 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.506+0000 7fa3f1590640 1 Processor -- start 2024-05-11T01:01:05.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.506+0000 7fa3f1590640 1 -- start start 2024-05-11T01:01:05.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3f1590640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3ec0b64c0 0x7fa3ec13f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3f1590640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 0x7fa3ec1e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3f1590640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3ec142a30 0x7fa3ec1e9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3f1590640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec074000 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3f1590640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec073e80 con 0x7fa3ec142a30 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3f1590640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ec0d06f0 con 0x7fa3ec0b64c0 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 0x7fa3ec1e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 0x7fa3ec1e8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60340/0 (socket says 172.21.15.52:60340) 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e27fc640 1 -- 172.21.15.52:0/187093931 learned_addr learned my addr 172.21.15.52:0/187093931 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3eaffd640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3ec0b64c0 0x7fa3ec13f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3eb7fe640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3ec142a30 0x7fa3ec1e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e27fc640 1 -- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3ec0b64c0 msgr2=0x7fa3ec13f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e27fc640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3ec0b64c0 0x7fa3ec13f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e27fc640 1 -- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3ec142a30 msgr2=0x7fa3ec1e9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e27fc640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3ec142a30 0x7fa3ec1e9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e27fc640 1 -- 172.21.15.52:0/187093931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec1f8ca0 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e27fc640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 0x7fa3ec1e8f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400d800 tx=0x7fa3d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:05.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e3fff640 1 -- 172.21.15.52:0/187093931 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400b6c0 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e3fff640 1 -- 172.21.15.52:0/187093931 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d4019070 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3e3fff640 1 -- 172.21.15.52:0/187093931 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400bac0 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ec1f8f40 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.507+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ec13f730 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.508+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ec069780 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.509+0000 7fa3e3fff640 1 -- 172.21.15.52:0/187093931 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa3d400bc60 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.509+0000 7fa3e3fff640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3cc0762d0 0x7fa3cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.510+0000 7fa3e3fff640 1 -- 172.21.15.52:0/187093931 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3d40979a0 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.510+0000 7fa3eaffd640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3cc0762d0 0x7fa3cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.510+0000 7fa3eaffd640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3cc0762d0 0x7fa3cc078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec1e9af0 tx=0x7fa3dc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.514+0000 7fa3e3fff640 1 -- 172.21.15.52:0/187093931 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa3d4060cb0 con 0x7fa3ec0cfd80 2024-05-11T01:01:05.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.702+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3ec06ffe0 con 0x7fa3cc0762d0 2024-05-11T01:01:05.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.709+0000 7fa3e3fff640 1 -- 172.21.15.52:0/187093931 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240868 (secure 0 0 0) 0x7fa3ec06ffe0 con 0x7fa3cc0762d0 2024-05-11T01:01:05.712 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:05.713 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:01:05.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3cc0762d0 msgr2=0x7fa3cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3cc0762d0 0x7fa3cc078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec1e9af0 tx=0x7fa3dc009040 comp rx=0 tx=0).stop 2024-05-11T01:01:05.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 msgr2=0x7fa3ec1e8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 0x7fa3ec1e8f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400d800 tx=0x7fa3d400dcc0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 shutdown_connections 2024-05-11T01:01:05.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3cc0762d0 0x7fa3cc078790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3ec142a30 0x7fa3ec1e9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec0cfd80 0x7fa3ec1e8f50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 --2- 172.21.15.52:0/187093931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3ec0b64c0 0x7fa3ec13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 >> 172.21.15.52:0/187093931 conn(0x7fa3ec0b3af0 msgr2=0x7fa3ec0d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:05.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 shutdown_connections 2024-05-11T01:01:05.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.712+0000 7fa3f1590640 1 -- 172.21.15.52:0/187093931 wait complete. 2024-05-11T01:01:05.724 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-05-11T01:01:05.510926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":2460,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":2519040,"data_stored":1585351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.845534"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672557+0000","last_change":"2024-05-11T01:01:00.672557+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672557+0000","last_undegraded":"2024-05-11T01:01:00.672557+0000","last_fullsized":"2024-05-11T01:01:00.672557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.670954+0000","last_change":"2024-05-11T01:01:00.670954+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.670954+0000","last_undegraded":"2024-05-11T01:01:00.670954+0000","last_fullsized":"2024-05-11T01:01:00.670954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.671724+0000","last_change":"2024-05-11T01:01:00.671724+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.671724+0000","last_undegraded":"2024-05-11T01:01:00.671724+0000","last_fullsized":"2024-05-11T01:01:00.671724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672102+0000","last_change":"2024-05-11T01:01:00.672102+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672102+0000","last_undegraded":"2024-05-11T01:01:00.672102+0000","last_fullsized":"2024-05-11T01:01:00.672102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672076+0000","last_change":"2024-05-11T01:01:00.672076+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672076+0000","last_undegraded":"2024-05-11T01:01:00.672076+0000","last_fullsized":"2024-05-11T01:01:00.672076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663676+0000","last_change":"2024-05-11T01:00:59.663676+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663676+0000","last_undegraded":"2024-05-11T01:00:59.663676+0000","last_fullsized":"2024-05-11T01:00:59.663676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685833+0000","last_change":"2024-05-11T01:01:00.684578+0000","last_active":"2024-05-11T01:01:00.685833+0000","last_peered":"2024-05-11T01:01:00.685833+0000","last_clean":"2024-05-11T01:01:00.685833+0000","last_became_active":"2024-05-11T01:01:00.684303+0000","last_became_peered":"2024-05-11T01:01:00.684303+0000","last_unstale":"2024-05-11T01:01:00.685833+0000","last_undegraded":"2024-05-11T01:01:00.685833+0000","last_fullsized":"2024-05-11T01:01:00.685833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:24:23.590323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663629+0000","last_change":"2024-05-11T01:00:59.663629+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663629+0000","last_undegraded":"2024-05-11T01:00:59.663629+0000","last_fullsized":"2024-05-11T01:00:59.663629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666535+0000","last_change":"2024-05-11T01:00:59.666535+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666535+0000","last_undegraded":"2024-05-11T01:00:59.666535+0000","last_fullsized":"2024-05-11T01:00:59.666535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.683064+0000","last_change":"2024-05-11T01:01:00.679681+0000","last_active":"2024-05-11T01:01:00.683064+0000","last_peered":"2024-05-11T01:01:00.683064+0000","last_clean":"2024-05-11T01:01:00.683064+0000","last_became_active":"2024-05-11T01:01:00.679125+0000","last_became_peered":"2024-05-11T01:01:00.679125+0000","last_unstale":"2024-05-11T01:01:00.683064+0000","last_undegraded":"2024-05-11T01:01:00.683064+0000","last_fullsized":"2024-05-11T01:01:00.683064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:13.659371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.668251+0000","last_change":"2024-05-11T01:00:59.668251+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.668251+0000","last_undegraded":"2024-05-11T01:00:59.668251+0000","last_fullsized":"2024-05-11T01:00:59.668251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666997+0000","last_change":"2024-05-11T01:00:59.666997+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666997+0000","last_undegraded":"2024-05-11T01:00:59.666997+0000","last_fullsized":"2024-05-11T01:00:59.666997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685683+0000","last_change":"2024-05-11T01:01:00.683575+0000","last_active":"2024-05-11T01:01:00.685683+0000","last_peered":"2024-05-11T01:01:00.685683+0000","last_clean":"2024-05-11T01:01:00.685683+0000","last_became_active":"2024-05-11T01:01:00.683259+0000","last_became_peered":"2024-05-11T01:01:00.683259+0000","last_unstale":"2024-05-11T01:01:00.685683+0000","last_undegraded":"2024-05-11T01:01:00.685683+0000","last_fullsized":"2024-05-11T01:01:00.685683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:38:37.156429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666516+0000","last_change":"2024-05-11T01:00:59.666516+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666516+0000","last_undegraded":"2024-05-11T01:00:59.666516+0000","last_fullsized":"2024-05-11T01:00:59.666516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.668583+0000","last_change":"2024-05-11T01:00:59.668583+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.668583+0000","last_undegraded":"2024-05-11T01:00:59.668583+0000","last_fullsized":"2024-05-11T01:00:59.668583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685870+0000","last_change":"2024-05-11T01:01:00.684714+0000","last_active":"2024-05-11T01:01:00.685870+0000","last_peered":"2024-05-11T01:01:00.685870+0000","last_clean":"2024-05-11T01:01:00.685870+0000","last_became_active":"2024-05-11T01:01:00.684454+0000","last_became_peered":"2024-05-11T01:01:00.684454+0000","last_unstale":"2024-05-11T01:01:00.685870+0000","last_undegraded":"2024-05-11T01:01:00.685870+0000","last_fullsized":"2024-05-11T01:01:00.685870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:12:31.764619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.670497+0000","last_change":"2024-05-11T01:01:00.670497+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.670497+0000","last_undegraded":"2024-05-11T01:01:00.670497+0000","last_fullsized":"2024-05-11T01:01:00.670497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669674+0000","last_change":"2024-05-11T01:00:59.669674+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669674+0000","last_undegraded":"2024-05-11T01:00:59.669674+0000","last_fullsized":"2024-05-11T01:00:59.669674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.667366+0000","last_change":"2024-05-11T01:00:59.667366+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.667366+0000","last_undegraded":"2024-05-11T01:00:59.667366+0000","last_fullsized":"2024-05-11T01:00:59.667366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669793+0000","last_change":"2024-05-11T01:00:59.669793+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669793+0000","last_undegraded":"2024-05-11T01:00:59.669793+0000","last_fullsized":"2024-05-11T01:00:59.669793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666432+0000","last_change":"2024-05-11T01:00:59.666432+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666432+0000","last_undegraded":"2024-05-11T01:00:59.666432+0000","last_fullsized":"2024-05-11T01:00:59.666432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669707+0000","last_change":"2024-05-11T01:00:59.669707+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669707+0000","last_undegraded":"2024-05-11T01:00:59.669707+0000","last_fullsized":"2024-05-11T01:00:59.669707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":104,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T01:00:59.668655+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:00:59.668655+0000","last_peered":"2024-05-11T01:00:59.668655+0000","last_clean":"2024-05-11T01:00:59.668655+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:00:59.668655+0000","last_undegraded":"2024-05-11T01:00:59.668655+0000","last_fullsized":"2024-05-11T01:00:59.668655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666394+0000","last_change":"2024-05-11T01:00:59.666394+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666394+0000","last_undegraded":"2024-05-11T01:00:59.666394+0000","last_fullsized":"2024-05-11T01:00:59.666394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663893+0000","last_change":"2024-05-11T01:00:59.663893+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663893+0000","last_undegraded":"2024-05-11T01:00:59.663893+0000","last_fullsized":"2024-05-11T01:00:59.663893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.670085+0000","last_change":"2024-05-11T01:00:59.670085+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.670085+0000","last_undegraded":"2024-05-11T01:00:59.670085+0000","last_fullsized":"2024-05-11T01:00:59.670085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666499+0000","last_change":"2024-05-11T01:00:59.666499+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666499+0000","last_undegraded":"2024-05-11T01:00:59.666499+0000","last_fullsized":"2024-05-11T01:00:59.666499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666195+0000","last_change":"2024-05-11T01:00:59.666195+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666195+0000","last_undegraded":"2024-05-11T01:00:59.666195+0000","last_fullsized":"2024-05-11T01:00:59.666195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663953+0000","last_change":"2024-05-11T01:00:59.663953+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663953+0000","last_undegraded":"2024-05-11T01:00:59.663953+0000","last_fullsized":"2024-05-11T01:00:59.663953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666984+0000","last_change":"2024-05-11T01:00:59.666984+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666984+0000","last_undegraded":"2024-05-11T01:00:59.666984+0000","last_fullsized":"2024-05-11T01:00:59.666984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.687089+0000","last_change":"2024-05-11T01:01:00.686048+0000","last_active":"2024-05-11T01:01:00.687089+0000","last_peered":"2024-05-11T01:01:00.687089+0000","last_clean":"2024-05-11T01:01:00.687089+0000","last_became_active":"2024-05-11T01:01:00.685215+0000","last_became_peered":"2024-05-11T01:01:00.685215+0000","last_unstale":"2024-05-11T01:01:00.687089+0000","last_undegraded":"2024-05-11T01:01:00.687089+0000","last_fullsized":"2024-05-11T01:01:00.687089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:00:17.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.665681+0000","last_change":"2024-05-11T01:00:59.665681+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.665681+0000","last_undegraded":"2024-05-11T01:00:59.665681+0000","last_fullsized":"2024-05-11T01:00:59.665681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666285+0000","last_change":"2024-05-11T01:00:59.666285+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666285+0000","last_undegraded":"2024-05-11T01:00:59.666285+0000","last_fullsized":"2024-05-11T01:00:59.666285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666055+0000","last_change":"2024-05-11T01:00:59.666055+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666055+0000","last_undegraded":"2024-05-11T01:00:59.666055+0000","last_fullsized":"2024-05-11T01:00:59.666055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663916+0000","last_change":"2024-05-11T01:00:59.663916+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663916+0000","last_undegraded":"2024-05-11T01:00:59.663916+0000","last_fullsized":"2024-05-11T01:00:59.663916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.671239+0000","last_change":"2024-05-11T01:01:00.671239+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.671239+0000","last_undegraded":"2024-05-11T01:01:00.671239+0000","last_fullsized":"2024-05-11T01:01:00.671239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.664011+0000","last_change":"2024-05-11T01:00:59.664011+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.664011+0000","last_undegraded":"2024-05-11T01:00:59.664011+0000","last_fullsized":"2024-05-11T01:00:59.664011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.664671+0000","last_change":"2024-05-11T01:00:59.664671+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.664671+0000","last_undegraded":"2024-05-11T01:00:59.664671+0000","last_fullsized":"2024-05-11T01:00:59.664671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663635+0000","last_change":"2024-05-11T01:00:59.663635+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663635+0000","last_undegraded":"2024-05-11T01:00:59.663635+0000","last_fullsized":"2024-05-11T01:00:59.663635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.667046+0000","last_change":"2024-05-11T01:00:59.667046+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.667046+0000","last_undegraded":"2024-05-11T01:00:59.667046+0000","last_fullsized":"2024-05-11T01:00:59.667046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":761856,"data_stored":641672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":761856,"data_stored":641672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:01:05.728 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:01:05.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.867+0000 7fd00e4a7640 1 Processor -- start 2024-05-11T01:01:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.867+0000 7fd00e4a7640 1 -- start start 2024-05-11T01:01:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.867+0000 7fd00e4a7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd008160480 0x7fd0080d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.867+0000 7fd00e4a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd0080d32c0 0x7fd0080d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.867+0000 7fd00e4a7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd008163250 con 0x7fd00815f430 2024-05-11T01:01:05.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.867+0000 7fd00e4a7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd008163530 con 0x7fd008160480 2024-05-11T01:01:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.867+0000 7fd00e4a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd008163810 con 0x7fd0080d32c0 2024-05-11T01:01:05.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd00ca44640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd0080d32c0 0x7fd0080d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd0077fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd008160480 0x7fd0080d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd00ca44640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd0080d32c0 0x7fd0080d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:51984/0 (socket says 172.21.15.52:51984) 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd0077fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd008160480 0x7fd0080d2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:45846/0 (socket says 172.21.15.52:45846) 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd00ca44640 1 -- 172.21.15.52:0/1860837377 learned_addr learned my addr 172.21.15.52:0/1860837377 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd00ca44640 1 -- 172.21.15.52:0/1860837377 >> v2:172.21.15.119:3300/0 conn(0x7fd008160480 msgr2=0x7fd0080d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd00ca44640 1 --2- 172.21.15.52:0/1860837377 >> v2:172.21.15.119:3300/0 conn(0x7fd008160480 0x7fd0080d2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd00ca44640 1 -- 172.21.15.52:0/1860837377 >> v1:172.21.15.119:6789/0 conn(0x7fd00815f430 legacy=0x7fd00815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd006ffd640 1 -- 172.21.15.52:0/1860837377 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3980680118 0 0) 0x7fd008163250 con 0x7fd00815f430 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd00ca44640 1 -- 172.21.15.52:0/1860837377 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd008163af0 con 0x7fd0080d32c0 2024-05-11T01:01:05.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.868+0000 7fd0077fe640 1 --2- 172.21.15.52:0/1860837377 >> v2:172.21.15.119:3300/0 conn(0x7fd008160480 0x7fd0080d2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.869+0000 7fd00ca44640 1 --2- 172.21.15.52:0/1860837377 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd0080d32c0 0x7fd0080d3710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd000009e50 tx=0x7fd00002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d94b22f842b2668 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:05.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.869+0000 7fd006ffd640 1 -- 172.21.15.52:0/1860837377 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000036070 con 0x7fd0080d32c0 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.869+0000 7fd006ffd640 1 -- 172.21.15.52:0/1860837377 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd000031600 con 0x7fd0080d32c0 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.869+0000 7fd006ffd640 1 -- 172.21.15.52:0/1860837377 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0000319d0 con 0x7fd0080d32c0 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.870+0000 7fd00e4a7640 1 -- 172.21.15.52:0/1860837377 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd0080d32c0 msgr2=0x7fd0080d3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.870+0000 7fd00e4a7640 1 --2- 172.21.15.52:0/1860837377 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd0080d32c0 0x7fd0080d3710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd000009e50 tx=0x7fd00002a710 comp rx=0 tx=0).stop 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.870+0000 7fd00e4a7640 1 -- 172.21.15.52:0/1860837377 shutdown_connections 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.870+0000 7fd00e4a7640 1 --2- 172.21.15.52:0/1860837377 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd0080d32c0 0x7fd0080d3710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.870+0000 7fd00e4a7640 1 --2- 172.21.15.52:0/1860837377 >> v2:172.21.15.119:3300/0 conn(0x7fd008160480 0x7fd0080d2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.870+0000 7fd00e4a7640 1 -- 172.21.15.52:0/1860837377 >> 172.21.15.52:0/1860837377 conn(0x7fd00808eb70 msgr2=0x7fd00809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.870+0000 7fd00e4a7640 1 -- 172.21.15.52:0/1860837377 shutdown_connections 2024-05-11T01:01:05.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.870+0000 7fd00e4a7640 1 -- 172.21.15.52:0/1860837377 wait complete. 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.871+0000 7fd00e4a7640 1 Processor -- start 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.871+0000 7fd00e4a7640 1 -- start start 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00e4a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0080d32c0 0x7fd0081569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00e4a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd00815f430 0x7fd00815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00e4a7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd008160480 0x7fd00815d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00e4a7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd008163030 con 0x7fd008160480 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00e4a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd008162eb0 con 0x7fd0080d32c0 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00e4a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd0081608f0 con 0x7fd00815f430 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00ca44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd008160480 0x7fd00815d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00ca44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd008160480 0x7fd00815d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60350/0 (socket says 172.21.15.52:60350) 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00ca44640 1 -- 172.21.15.52:0/477108604 learned_addr learned my addr 172.21.15.52:0/477108604 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd007fff640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0080d32c0 0x7fd0081569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd0077fe640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd00815f430 0x7fd00815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00ca44640 1 -- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd00815f430 msgr2=0x7fd00815d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00ca44640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd00815f430 0x7fd00815d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00ca44640 1 -- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0080d32c0 msgr2=0x7fd0081569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00ca44640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0080d32c0 0x7fd0081569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd00ca44640 1 -- 172.21.15.52:0/477108604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd008156ee0 con 0x7fd008160480 2024-05-11T01:01:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd0077fe640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd00815f430 0x7fd00815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.872+0000 7fd007fff640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0080d32c0 0x7fd0081569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.873+0000 7fd00ca44640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd008160480 0x7fd00815d650 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd000034040 tx=0x7fd000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.873+0000 7fd004ff9640 1 -- 172.21.15.52:0/477108604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000041020 con 0x7fd008160480 2024-05-11T01:01:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.873+0000 7fd004ff9640 1 -- 172.21.15.52:0/477108604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd000037040 con 0x7fd008160480 2024-05-11T01:01:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.873+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0080cd220 con 0x7fd008160480 2024-05-11T01:01:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.873+0000 7fd004ff9640 1 -- 172.21.15.52:0/477108604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000036050 con 0x7fd008160480 2024-05-11T01:01:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.873+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0080cc740 con 0x7fd008160480 2024-05-11T01:01:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.874+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd008069780 con 0x7fd008160480 2024-05-11T01:01:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.874+0000 7fd004ff9640 1 -- 172.21.15.52:0/477108604 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd000007c50 con 0x7fd008160480 2024-05-11T01:01:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.875+0000 7fd004ff9640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcff40762d0 0x7fcff4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.875+0000 7fd004ff9640 1 -- 172.21.15.52:0/477108604 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd0000b7cb0 con 0x7fd008160480 2024-05-11T01:01:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.875+0000 7fd007fff640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcff40762d0 0x7fcff4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.876+0000 7fd007fff640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcff40762d0 0x7fcff4078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcff8000f60 tx=0x7fcff8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:05.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:05.879+0000 7fd004ff9640 1 -- 172.21.15.52:0/477108604 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd00004b060 con 0x7fd008160480 2024-05-11T01:01:06.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.066+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0080cc210 con 0x7fcff40762d0 2024-05-11T01:01:06.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.072+0000 7fd004ff9640 1 -- 172.21.15.52:0/477108604 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240868 (secure 0 0 0) 0x7fd0080cc210 con 0x7fcff40762d0 2024-05-11T01:01:06.075 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:06.076 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:01:06.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcff40762d0 msgr2=0x7fcff4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcff40762d0 0x7fcff4078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcff8000f60 tx=0x7fcff8009040 comp rx=0 tx=0).stop 2024-05-11T01:01:06.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd008160480 msgr2=0x7fd00815d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd008160480 0x7fd00815d650 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd000034040 tx=0x7fd000004a40 comp rx=0 tx=0).stop 2024-05-11T01:01:06.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 shutdown_connections 2024-05-11T01:01:06.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcff40762d0 0x7fcff4078790 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd008160480 0x7fd00815d650 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd00815f430 0x7fd00815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 --2- 172.21.15.52:0/477108604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0080d32c0 0x7fd0081569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 >> 172.21.15.52:0/477108604 conn(0x7fd00808eb70 msgr2=0x7fd00809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:06.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 shutdown_connections 2024-05-11T01:01:06.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.075+0000 7fd00e4a7640 1 -- 172.21.15.52:0/477108604 wait complete. 2024-05-11T01:01:06.087 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-05-11T01:01:05.510926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":2460,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":2519040,"data_stored":1585351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.845534"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672557+0000","last_change":"2024-05-11T01:01:00.672557+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672557+0000","last_undegraded":"2024-05-11T01:01:00.672557+0000","last_fullsized":"2024-05-11T01:01:00.672557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.670954+0000","last_change":"2024-05-11T01:01:00.670954+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.670954+0000","last_undegraded":"2024-05-11T01:01:00.670954+0000","last_fullsized":"2024-05-11T01:01:00.670954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.671724+0000","last_change":"2024-05-11T01:01:00.671724+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.671724+0000","last_undegraded":"2024-05-11T01:01:00.671724+0000","last_fullsized":"2024-05-11T01:01:00.671724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672102+0000","last_change":"2024-05-11T01:01:00.672102+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672102+0000","last_undegraded":"2024-05-11T01:01:00.672102+0000","last_fullsized":"2024-05-11T01:01:00.672102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672076+0000","last_change":"2024-05-11T01:01:00.672076+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672076+0000","last_undegraded":"2024-05-11T01:01:00.672076+0000","last_fullsized":"2024-05-11T01:01:00.672076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663676+0000","last_change":"2024-05-11T01:00:59.663676+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663676+0000","last_undegraded":"2024-05-11T01:00:59.663676+0000","last_fullsized":"2024-05-11T01:00:59.663676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685833+0000","last_change":"2024-05-11T01:01:00.684578+0000","last_active":"2024-05-11T01:01:00.685833+0000","last_peered":"2024-05-11T01:01:00.685833+0000","last_clean":"2024-05-11T01:01:00.685833+0000","last_became_active":"2024-05-11T01:01:00.684303+0000","last_became_peered":"2024-05-11T01:01:00.684303+0000","last_unstale":"2024-05-11T01:01:00.685833+0000","last_undegraded":"2024-05-11T01:01:00.685833+0000","last_fullsized":"2024-05-11T01:01:00.685833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:24:23.590323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663629+0000","last_change":"2024-05-11T01:00:59.663629+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663629+0000","last_undegraded":"2024-05-11T01:00:59.663629+0000","last_fullsized":"2024-05-11T01:00:59.663629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666535+0000","last_change":"2024-05-11T01:00:59.666535+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666535+0000","last_undegraded":"2024-05-11T01:00:59.666535+0000","last_fullsized":"2024-05-11T01:00:59.666535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.683064+0000","last_change":"2024-05-11T01:01:00.679681+0000","last_active":"2024-05-11T01:01:00.683064+0000","last_peered":"2024-05-11T01:01:00.683064+0000","last_clean":"2024-05-11T01:01:00.683064+0000","last_became_active":"2024-05-11T01:01:00.679125+0000","last_became_peered":"2024-05-11T01:01:00.679125+0000","last_unstale":"2024-05-11T01:01:00.683064+0000","last_undegraded":"2024-05-11T01:01:00.683064+0000","last_fullsized":"2024-05-11T01:01:00.683064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:13.659371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.668251+0000","last_change":"2024-05-11T01:00:59.668251+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.668251+0000","last_undegraded":"2024-05-11T01:00:59.668251+0000","last_fullsized":"2024-05-11T01:00:59.668251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666997+0000","last_change":"2024-05-11T01:00:59.666997+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666997+0000","last_undegraded":"2024-05-11T01:00:59.666997+0000","last_fullsized":"2024-05-11T01:00:59.666997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685683+0000","last_change":"2024-05-11T01:01:00.683575+0000","last_active":"2024-05-11T01:01:00.685683+0000","last_peered":"2024-05-11T01:01:00.685683+0000","last_clean":"2024-05-11T01:01:00.685683+0000","last_became_active":"2024-05-11T01:01:00.683259+0000","last_became_peered":"2024-05-11T01:01:00.683259+0000","last_unstale":"2024-05-11T01:01:00.685683+0000","last_undegraded":"2024-05-11T01:01:00.685683+0000","last_fullsized":"2024-05-11T01:01:00.685683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:38:37.156429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666516+0000","last_change":"2024-05-11T01:00:59.666516+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666516+0000","last_undegraded":"2024-05-11T01:00:59.666516+0000","last_fullsized":"2024-05-11T01:00:59.666516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.668583+0000","last_change":"2024-05-11T01:00:59.668583+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.668583+0000","last_undegraded":"2024-05-11T01:00:59.668583+0000","last_fullsized":"2024-05-11T01:00:59.668583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685870+0000","last_change":"2024-05-11T01:01:00.684714+0000","last_active":"2024-05-11T01:01:00.685870+0000","last_peered":"2024-05-11T01:01:00.685870+0000","last_clean":"2024-05-11T01:01:00.685870+0000","last_became_active":"2024-05-11T01:01:00.684454+0000","last_became_peered":"2024-05-11T01:01:00.684454+0000","last_unstale":"2024-05-11T01:01:00.685870+0000","last_undegraded":"2024-05-11T01:01:00.685870+0000","last_fullsized":"2024-05-11T01:01:00.685870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:12:31.764619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.670497+0000","last_change":"2024-05-11T01:01:00.670497+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.670497+0000","last_undegraded":"2024-05-11T01:01:00.670497+0000","last_fullsized":"2024-05-11T01:01:00.670497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669674+0000","last_change":"2024-05-11T01:00:59.669674+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669674+0000","last_undegraded":"2024-05-11T01:00:59.669674+0000","last_fullsized":"2024-05-11T01:00:59.669674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.667366+0000","last_change":"2024-05-11T01:00:59.667366+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.667366+0000","last_undegraded":"2024-05-11T01:00:59.667366+0000","last_fullsized":"2024-05-11T01:00:59.667366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669793+0000","last_change":"2024-05-11T01:00:59.669793+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669793+0000","last_undegraded":"2024-05-11T01:00:59.669793+0000","last_fullsized":"2024-05-11T01:00:59.669793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666432+0000","last_change":"2024-05-11T01:00:59.666432+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666432+0000","last_undegraded":"2024-05-11T01:00:59.666432+0000","last_fullsized":"2024-05-11T01:00:59.666432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669707+0000","last_change":"2024-05-11T01:00:59.669707+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669707+0000","last_undegraded":"2024-05-11T01:00:59.669707+0000","last_fullsized":"2024-05-11T01:00:59.669707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":104,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T01:00:59.668655+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:00:59.668655+0000","last_peered":"2024-05-11T01:00:59.668655+0000","last_clean":"2024-05-11T01:00:59.668655+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:00:59.668655+0000","last_undegraded":"2024-05-11T01:00:59.668655+0000","last_fullsized":"2024-05-11T01:00:59.668655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666394+0000","last_change":"2024-05-11T01:00:59.666394+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666394+0000","last_undegraded":"2024-05-11T01:00:59.666394+0000","last_fullsized":"2024-05-11T01:00:59.666394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663893+0000","last_change":"2024-05-11T01:00:59.663893+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663893+0000","last_undegraded":"2024-05-11T01:00:59.663893+0000","last_fullsized":"2024-05-11T01:00:59.663893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.670085+0000","last_change":"2024-05-11T01:00:59.670085+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.670085+0000","last_undegraded":"2024-05-11T01:00:59.670085+0000","last_fullsized":"2024-05-11T01:00:59.670085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666499+0000","last_change":"2024-05-11T01:00:59.666499+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666499+0000","last_undegraded":"2024-05-11T01:00:59.666499+0000","last_fullsized":"2024-05-11T01:00:59.666499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666195+0000","last_change":"2024-05-11T01:00:59.666195+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666195+0000","last_undegraded":"2024-05-11T01:00:59.666195+0000","last_fullsized":"2024-05-11T01:00:59.666195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663953+0000","last_change":"2024-05-11T01:00:59.663953+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663953+0000","last_undegraded":"2024-05-11T01:00:59.663953+0000","last_fullsized":"2024-05-11T01:00:59.663953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666984+0000","last_change":"2024-05-11T01:00:59.666984+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666984+0000","last_undegraded":"2024-05-11T01:00:59.666984+0000","last_fullsized":"2024-05-11T01:00:59.666984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.687089+0000","last_change":"2024-05-11T01:01:00.686048+0000","last_active":"2024-05-11T01:01:00.687089+0000","last_peered":"2024-05-11T01:01:00.687089+0000","last_clean":"2024-05-11T01:01:00.687089+0000","last_became_active":"2024-05-11T01:01:00.685215+0000","last_became_peered":"2024-05-11T01:01:00.685215+0000","last_unstale":"2024-05-11T01:01:00.687089+0000","last_undegraded":"2024-05-11T01:01:00.687089+0000","last_fullsized":"2024-05-11T01:01:00.687089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:00:17.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.665681+0000","last_change":"2024-05-11T01:00:59.665681+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.665681+0000","last_undegraded":"2024-05-11T01:00:59.665681+0000","last_fullsized":"2024-05-11T01:00:59.665681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666285+0000","last_change":"2024-05-11T01:00:59.666285+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666285+0000","last_undegraded":"2024-05-11T01:00:59.666285+0000","last_fullsized":"2024-05-11T01:00:59.666285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666055+0000","last_change":"2024-05-11T01:00:59.666055+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666055+0000","last_undegraded":"2024-05-11T01:00:59.666055+0000","last_fullsized":"2024-05-11T01:00:59.666055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663916+0000","last_change":"2024-05-11T01:00:59.663916+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663916+0000","last_undegraded":"2024-05-11T01:00:59.663916+0000","last_fullsized":"2024-05-11T01:00:59.663916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.671239+0000","last_change":"2024-05-11T01:01:00.671239+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.671239+0000","last_undegraded":"2024-05-11T01:01:00.671239+0000","last_fullsized":"2024-05-11T01:01:00.671239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.664011+0000","last_change":"2024-05-11T01:00:59.664011+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.664011+0000","last_undegraded":"2024-05-11T01:00:59.664011+0000","last_fullsized":"2024-05-11T01:00:59.664011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.664671+0000","last_change":"2024-05-11T01:00:59.664671+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.664671+0000","last_undegraded":"2024-05-11T01:00:59.664671+0000","last_fullsized":"2024-05-11T01:00:59.664671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663635+0000","last_change":"2024-05-11T01:00:59.663635+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663635+0000","last_undegraded":"2024-05-11T01:00:59.663635+0000","last_fullsized":"2024-05-11T01:00:59.663635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.667046+0000","last_change":"2024-05-11T01:00:59.667046+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.667046+0000","last_undegraded":"2024-05-11T01:00:59.667046+0000","last_fullsized":"2024-05-11T01:00:59.667046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":761856,"data_stored":641672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":761856,"data_stored":641672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:01:06.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:01:06.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.231+0000 7f576d059640 1 Processor -- start 2024-05-11T01:01:06.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.231+0000 7f576d059640 1 -- start start 2024-05-11T01:01:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.231+0000 7f576d059640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 0x7f57680c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f576d059640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57680d0d80 0x7f57680d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f576d059640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57680d02e0 con 0x7f5768160730 2024-05-11T01:01:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f576d059640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f57680d08f0 con 0x7f57680d0d80 2024-05-11T01:01:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f576d059640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f57680c7d50 con 0x7f57680b1260 2024-05-11T01:01:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f576759e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57680d0d80 0x7f57680d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f5766d9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 0x7f57680c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f575ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5768160730 0x7f576806fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:37844/0 (socket says 172.21.15.52:37844) 2024-05-11T01:01:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f575ffff640 1 -- 172.21.15.52:0/1336714180 learned_addr learned my addr 172.21.15.52:0/1336714180 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f576759e640 1 -- 172.21.15.52:0/1336714180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 msgr2=0x7f57680c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f576759e640 1 --2- 172.21.15.52:0/1336714180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 0x7f57680c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f576759e640 1 -- 172.21.15.52:0/1336714180 >> v1:172.21.15.119:6789/0 conn(0x7f5768160730 legacy=0x7f576806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.232+0000 7f576759e640 1 -- 172.21.15.52:0/1336714180 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57680c8030 con 0x7f57680d0d80 2024-05-11T01:01:06.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.233+0000 7f576759e640 1 --2- 172.21.15.52:0/1336714180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57680d0d80 0x7f57680d1210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5760009d90 tx=0x7f576002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cbee2265cec826 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:06.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.233+0000 7f576659c640 1 -- 172.21.15.52:0/1336714180 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760007740 con 0x7f57680d0d80 2024-05-11T01:01:06.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.233+0000 7f576659c640 1 -- 172.21.15.52:0/1336714180 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57600078e0 con 0x7f57680d0d80 2024-05-11T01:01:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.233+0000 7f576659c640 1 -- 172.21.15.52:0/1336714180 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760007cb0 con 0x7f57680d0d80 2024-05-11T01:01:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.234+0000 7f576d059640 1 -- 172.21.15.52:0/1336714180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57680d0d80 msgr2=0x7f57680d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.234+0000 7f576d059640 1 --2- 172.21.15.52:0/1336714180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57680d0d80 0x7f57680d1210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5760009d90 tx=0x7f576002a660 comp rx=0 tx=0).stop 2024-05-11T01:01:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.234+0000 7f576d059640 1 -- 172.21.15.52:0/1336714180 shutdown_connections 2024-05-11T01:01:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.234+0000 7f576d059640 1 --2- 172.21.15.52:0/1336714180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57680d0d80 0x7f57680d1210 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.234+0000 7f576d059640 1 --2- 172.21.15.52:0/1336714180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 0x7f57680c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.234+0000 7f576d059640 1 -- 172.21.15.52:0/1336714180 >> 172.21.15.52:0/1336714180 conn(0x7f57680b3a10 msgr2=0x7f57680b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.234+0000 7f576d059640 1 -- 172.21.15.52:0/1336714180 shutdown_connections 2024-05-11T01:01:06.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.234+0000 7f576d059640 1 -- 172.21.15.52:0/1336714180 wait complete. 2024-05-11T01:01:06.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.235+0000 7f576d059640 1 Processor -- start 2024-05-11T01:01:06.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.235+0000 7f576d059640 1 -- start start 2024-05-11T01:01:06.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f576d059640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 0x7f576814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f576d059640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57680d0d80 0x7f576815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f576d059640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768160730 0x7f576815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f576d059640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5768160b90 con 0x7f57680d0d80 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f576d059640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5768070470 con 0x7f5768160730 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f576d059640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f57680b1640 con 0x7f57680b1260 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57680d0d80 0x7f576815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57680d0d80 0x7f576815d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60356/0 (socket says 172.21.15.52:60356) 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575ffff640 1 -- 172.21.15.52:0/322893487 learned_addr learned my addr 172.21.15.52:0/322893487 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f5766d9d640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 0x7f576814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f576759e640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768160730 0x7f576815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575ffff640 1 -- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 msgr2=0x7f576814c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575ffff640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 0x7f576814c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575ffff640 1 -- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768160730 msgr2=0x7f576815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575ffff640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768160730 0x7f576815da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575ffff640 1 -- 172.21.15.52:0/322893487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576815e260 con 0x7f57680d0d80 2024-05-11T01:01:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575ffff640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57680d0d80 0x7f576815d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f575000c500 tx=0x7f575000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575f7fe640 1 -- 172.21.15.52:0/322893487 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750017070 con 0x7f57680d0d80 2024-05-11T01:01:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f575f7fe640 1 -- 172.21.15.52:0/322893487 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5750005d00 con 0x7f57680d0d80 2024-05-11T01:01:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.236+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576815e540 con 0x7f57680d0d80 2024-05-11T01:01:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.237+0000 7f575f7fe640 1 -- 172.21.15.52:0/322893487 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750007330 con 0x7f57680d0d80 2024-05-11T01:01:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.237+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576815eb00 con 0x7f57680d0d80 2024-05-11T01:01:06.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.238+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5768069780 con 0x7f57680d0d80 2024-05-11T01:01:06.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.238+0000 7f575f7fe640 1 -- 172.21.15.52:0/322893487 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f575001a070 con 0x7f57680d0d80 2024-05-11T01:01:06.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.239+0000 7f575f7fe640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f57480762d0 0x7f5748078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.239+0000 7f575f7fe640 1 -- 172.21.15.52:0/322893487 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f57500968a0 con 0x7f57680d0d80 2024-05-11T01:01:06.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.239+0000 7f5766d9d640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f57480762d0 0x7f5748078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.239+0000 7f5766d9d640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f57480762d0 0x7f5748078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5758009aa0 tx=0x7f5758009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:06.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.243+0000 7f575f7fe640 1 -- 172.21.15.52:0/322893487 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f575005fc40 con 0x7f57680d0d80 2024-05-11T01:01:06.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.431+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f576806f630 con 0x7f57480762d0 2024-05-11T01:01:06.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.438+0000 7f575f7fe640 1 -- 172.21.15.52:0/322893487 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240868 (secure 0 0 0) 0x7f576806f630 con 0x7f57480762d0 2024-05-11T01:01:06.441 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:01:06.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f57480762d0 msgr2=0x7f5748078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f57480762d0 0x7f5748078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5758009aa0 tx=0x7f5758009190 comp rx=0 tx=0).stop 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57680d0d80 msgr2=0x7f576815d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57680d0d80 0x7f576815d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f575000c500 tx=0x7f575000c9c0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 shutdown_connections 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f57480762d0 0x7f5748078790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5768160730 0x7f576815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57680d0d80 0x7f576815d4e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 --2- 172.21.15.52:0/322893487 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f57680b1260 0x7f576814c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 >> 172.21.15.52:0/322893487 conn(0x7f57680b3a10 msgr2=0x7f57680d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 shutdown_connections 2024-05-11T01:01:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.441+0000 7f576d059640 1 -- 172.21.15.52:0/322893487 wait complete. 2024-05-11T01:01:06.453 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-05-11T01:01:05.510926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":2460,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":2519040,"data_stored":1585351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.845534"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672557+0000","last_change":"2024-05-11T01:01:00.672557+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672557+0000","last_undegraded":"2024-05-11T01:01:00.672557+0000","last_fullsized":"2024-05-11T01:01:00.672557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.670954+0000","last_change":"2024-05-11T01:01:00.670954+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.670954+0000","last_undegraded":"2024-05-11T01:01:00.670954+0000","last_fullsized":"2024-05-11T01:01:00.670954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.671724+0000","last_change":"2024-05-11T01:01:00.671724+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.671724+0000","last_undegraded":"2024-05-11T01:01:00.671724+0000","last_fullsized":"2024-05-11T01:01:00.671724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672102+0000","last_change":"2024-05-11T01:01:00.672102+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672102+0000","last_undegraded":"2024-05-11T01:01:00.672102+0000","last_fullsized":"2024-05-11T01:01:00.672102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672076+0000","last_change":"2024-05-11T01:01:00.672076+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672076+0000","last_undegraded":"2024-05-11T01:01:00.672076+0000","last_fullsized":"2024-05-11T01:01:00.672076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663676+0000","last_change":"2024-05-11T01:00:59.663676+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663676+0000","last_undegraded":"2024-05-11T01:00:59.663676+0000","last_fullsized":"2024-05-11T01:00:59.663676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685833+0000","last_change":"2024-05-11T01:01:00.684578+0000","last_active":"2024-05-11T01:01:00.685833+0000","last_peered":"2024-05-11T01:01:00.685833+0000","last_clean":"2024-05-11T01:01:00.685833+0000","last_became_active":"2024-05-11T01:01:00.684303+0000","last_became_peered":"2024-05-11T01:01:00.684303+0000","last_unstale":"2024-05-11T01:01:00.685833+0000","last_undegraded":"2024-05-11T01:01:00.685833+0000","last_fullsized":"2024-05-11T01:01:00.685833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:24:23.590323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663629+0000","last_change":"2024-05-11T01:00:59.663629+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663629+0000","last_undegraded":"2024-05-11T01:00:59.663629+0000","last_fullsized":"2024-05-11T01:00:59.663629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666535+0000","last_change":"2024-05-11T01:00:59.666535+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666535+0000","last_undegraded":"2024-05-11T01:00:59.666535+0000","last_fullsized":"2024-05-11T01:00:59.666535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.683064+0000","last_change":"2024-05-11T01:01:00.679681+0000","last_active":"2024-05-11T01:01:00.683064+0000","last_peered":"2024-05-11T01:01:00.683064+0000","last_clean":"2024-05-11T01:01:00.683064+0000","last_became_active":"2024-05-11T01:01:00.679125+0000","last_became_peered":"2024-05-11T01:01:00.679125+0000","last_unstale":"2024-05-11T01:01:00.683064+0000","last_undegraded":"2024-05-11T01:01:00.683064+0000","last_fullsized":"2024-05-11T01:01:00.683064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:13.659371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.668251+0000","last_change":"2024-05-11T01:00:59.668251+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.668251+0000","last_undegraded":"2024-05-11T01:00:59.668251+0000","last_fullsized":"2024-05-11T01:00:59.668251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666997+0000","last_change":"2024-05-11T01:00:59.666997+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666997+0000","last_undegraded":"2024-05-11T01:00:59.666997+0000","last_fullsized":"2024-05-11T01:00:59.666997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685683+0000","last_change":"2024-05-11T01:01:00.683575+0000","last_active":"2024-05-11T01:01:00.685683+0000","last_peered":"2024-05-11T01:01:00.685683+0000","last_clean":"2024-05-11T01:01:00.685683+0000","last_became_active":"2024-05-11T01:01:00.683259+0000","last_became_peered":"2024-05-11T01:01:00.683259+0000","last_unstale":"2024-05-11T01:01:00.685683+0000","last_undegraded":"2024-05-11T01:01:00.685683+0000","last_fullsized":"2024-05-11T01:01:00.685683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:38:37.156429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666516+0000","last_change":"2024-05-11T01:00:59.666516+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666516+0000","last_undegraded":"2024-05-11T01:00:59.666516+0000","last_fullsized":"2024-05-11T01:00:59.666516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.668583+0000","last_change":"2024-05-11T01:00:59.668583+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.668583+0000","last_undegraded":"2024-05-11T01:00:59.668583+0000","last_fullsized":"2024-05-11T01:00:59.668583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685870+0000","last_change":"2024-05-11T01:01:00.684714+0000","last_active":"2024-05-11T01:01:00.685870+0000","last_peered":"2024-05-11T01:01:00.685870+0000","last_clean":"2024-05-11T01:01:00.685870+0000","last_became_active":"2024-05-11T01:01:00.684454+0000","last_became_peered":"2024-05-11T01:01:00.684454+0000","last_unstale":"2024-05-11T01:01:00.685870+0000","last_undegraded":"2024-05-11T01:01:00.685870+0000","last_fullsized":"2024-05-11T01:01:00.685870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:12:31.764619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.670497+0000","last_change":"2024-05-11T01:01:00.670497+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.670497+0000","last_undegraded":"2024-05-11T01:01:00.670497+0000","last_fullsized":"2024-05-11T01:01:00.670497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669674+0000","last_change":"2024-05-11T01:00:59.669674+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669674+0000","last_undegraded":"2024-05-11T01:00:59.669674+0000","last_fullsized":"2024-05-11T01:00:59.669674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.667366+0000","last_change":"2024-05-11T01:00:59.667366+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.667366+0000","last_undegraded":"2024-05-11T01:00:59.667366+0000","last_fullsized":"2024-05-11T01:00:59.667366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669793+0000","last_change":"2024-05-11T01:00:59.669793+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669793+0000","last_undegraded":"2024-05-11T01:00:59.669793+0000","last_fullsized":"2024-05-11T01:00:59.669793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666432+0000","last_change":"2024-05-11T01:00:59.666432+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666432+0000","last_undegraded":"2024-05-11T01:00:59.666432+0000","last_fullsized":"2024-05-11T01:00:59.666432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669707+0000","last_change":"2024-05-11T01:00:59.669707+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669707+0000","last_undegraded":"2024-05-11T01:00:59.669707+0000","last_fullsized":"2024-05-11T01:00:59.669707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":104,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T01:00:59.668655+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:00:59.668655+0000","last_peered":"2024-05-11T01:00:59.668655+0000","last_clean":"2024-05-11T01:00:59.668655+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:00:59.668655+0000","last_undegraded":"2024-05-11T01:00:59.668655+0000","last_fullsized":"2024-05-11T01:00:59.668655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666394+0000","last_change":"2024-05-11T01:00:59.666394+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666394+0000","last_undegraded":"2024-05-11T01:00:59.666394+0000","last_fullsized":"2024-05-11T01:00:59.666394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663893+0000","last_change":"2024-05-11T01:00:59.663893+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663893+0000","last_undegraded":"2024-05-11T01:00:59.663893+0000","last_fullsized":"2024-05-11T01:00:59.663893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.670085+0000","last_change":"2024-05-11T01:00:59.670085+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.670085+0000","last_undegraded":"2024-05-11T01:00:59.670085+0000","last_fullsized":"2024-05-11T01:00:59.670085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666499+0000","last_change":"2024-05-11T01:00:59.666499+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666499+0000","last_undegraded":"2024-05-11T01:00:59.666499+0000","last_fullsized":"2024-05-11T01:00:59.666499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666195+0000","last_change":"2024-05-11T01:00:59.666195+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666195+0000","last_undegraded":"2024-05-11T01:00:59.666195+0000","last_fullsized":"2024-05-11T01:00:59.666195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663953+0000","last_change":"2024-05-11T01:00:59.663953+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663953+0000","last_undegraded":"2024-05-11T01:00:59.663953+0000","last_fullsized":"2024-05-11T01:00:59.663953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666984+0000","last_change":"2024-05-11T01:00:59.666984+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666984+0000","last_undegraded":"2024-05-11T01:00:59.666984+0000","last_fullsized":"2024-05-11T01:00:59.666984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.687089+0000","last_change":"2024-05-11T01:01:00.686048+0000","last_active":"2024-05-11T01:01:00.687089+0000","last_peered":"2024-05-11T01:01:00.687089+0000","last_clean":"2024-05-11T01:01:00.687089+0000","last_became_active":"2024-05-11T01:01:00.685215+0000","last_became_peered":"2024-05-11T01:01:00.685215+0000","last_unstale":"2024-05-11T01:01:00.687089+0000","last_undegraded":"2024-05-11T01:01:00.687089+0000","last_fullsized":"2024-05-11T01:01:00.687089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:00:17.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.665681+0000","last_change":"2024-05-11T01:00:59.665681+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.665681+0000","last_undegraded":"2024-05-11T01:00:59.665681+0000","last_fullsized":"2024-05-11T01:00:59.665681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666285+0000","last_change":"2024-05-11T01:00:59.666285+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666285+0000","last_undegraded":"2024-05-11T01:00:59.666285+0000","last_fullsized":"2024-05-11T01:00:59.666285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666055+0000","last_change":"2024-05-11T01:00:59.666055+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666055+0000","last_undegraded":"2024-05-11T01:00:59.666055+0000","last_fullsized":"2024-05-11T01:00:59.666055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663916+0000","last_change":"2024-05-11T01:00:59.663916+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663916+0000","last_undegraded":"2024-05-11T01:00:59.663916+0000","last_fullsized":"2024-05-11T01:00:59.663916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.671239+0000","last_change":"2024-05-11T01:01:00.671239+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.671239+0000","last_undegraded":"2024-05-11T01:01:00.671239+0000","last_fullsized":"2024-05-11T01:01:00.671239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.664011+0000","last_change":"2024-05-11T01:00:59.664011+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.664011+0000","last_undegraded":"2024-05-11T01:00:59.664011+0000","last_fullsized":"2024-05-11T01:00:59.664011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.664671+0000","last_change":"2024-05-11T01:00:59.664671+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.664671+0000","last_undegraded":"2024-05-11T01:00:59.664671+0000","last_fullsized":"2024-05-11T01:00:59.664671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663635+0000","last_change":"2024-05-11T01:00:59.663635+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663635+0000","last_undegraded":"2024-05-11T01:00:59.663635+0000","last_fullsized":"2024-05-11T01:00:59.663635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.667046+0000","last_change":"2024-05-11T01:00:59.667046+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.667046+0000","last_undegraded":"2024-05-11T01:00:59.667046+0000","last_fullsized":"2024-05-11T01:00:59.667046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":761856,"data_stored":641672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":761856,"data_stored":641672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:01:06.456 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-05-11T01:01:06.457 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:01:00.672557+0000', 'last_change': '2024-05-11T01:01:00.672557+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.672557+0000', 'last_undegraded': '2024-05-11T01:01:00.672557+0000', 'last_fullsized': '2024-05-11T01:01:00.672557+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-05-11T01:01:06.458 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:01:00.670954+0000', 'last_change': '2024-05-11T01:01:00.670954+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.670954+0000', 'last_undegraded': '2024-05-11T01:01:00.670954+0000', 'last_fullsized': '2024-05-11T01:01:00.670954+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.459 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:01:00.671724+0000', 'last_change': '2024-05-11T01:01:00.671724+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.671724+0000', 'last_undegraded': '2024-05-11T01:01:00.671724+0000', 'last_fullsized': '2024-05-11T01:01:00.671724+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-05-11T01:01:06.460 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:01:00.672102+0000', 'last_change': '2024-05-11T01:01:00.672102+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.672102+0000', 'last_undegraded': '2024-05-11T01:01:00.672102+0000', 'last_fullsized': '2024-05-11T01:01:00.672102+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-05-11T01:01:06.461 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:01:00.672076+0000', 'last_change': '2024-05-11T01:01:00.672076+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.672076+0000', 'last_undegraded': '2024-05-11T01:01:00.672076+0000', 'last_fullsized': '2024-05-11T01:01:00.672076+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.663676+0000', 'last_change': '2024-05-11T01:00:59.663676+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.663676+0000', 'last_undegraded': '2024-05-11T01:00:59.663676+0000', 'last_fullsized': '2024-05-11T01:00:59.663676+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-05-11T01:01:06.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.663629+0000', 'last_change': '2024-05-11T01:00:59.663629+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.663629+0000', 'last_undegraded': '2024-05-11T01:00:59.663629+0000', 'last_fullsized': '2024-05-11T01:00:59.663629+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666535+0000', 'last_change': '2024-05-11T01:00:59.666535+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666535+0000', 'last_undegraded': '2024-05-11T01:00:59.666535+0000', 'last_fullsized': '2024-05-11T01:00:59.666535+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.463 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.668251+0000', 'last_change': '2024-05-11T01:00:59.668251+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.668251+0000', 'last_undegraded': '2024-05-11T01:00:59.668251+0000', 'last_fullsized': '2024-05-11T01:00:59.668251+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:PG 2.15 is not active+clean 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666997+0000', 'last_change': '2024-05-11T01:00:59.666997+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666997+0000', 'last_undegraded': '2024-05-11T01:00:59.666997+0000', 'last_fullsized': '2024-05-11T01:00:59.666997+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-05-11T01:01:06.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666516+0000', 'last_change': '2024-05-11T01:00:59.666516+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666516+0000', 'last_undegraded': '2024-05-11T01:00:59.666516+0000', 'last_fullsized': '2024-05-11T01:00:59.666516+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.668583+0000', 'last_change': '2024-05-11T01:00:59.668583+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.668583+0000', 'last_undegraded': '2024-05-11T01:00:59.668583+0000', 'last_fullsized': '2024-05-11T01:00:59.668583+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:01:00.670497+0000', 'last_change': '2024-05-11T01:01:00.670497+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.670497+0000', 'last_undegraded': '2024-05-11T01:01:00.670497+0000', 'last_fullsized': '2024-05-11T01:01:00.670497+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:PG 2.10 is not active+clean 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.669674+0000', 'last_change': '2024-05-11T01:00:59.669674+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.669674+0000', 'last_undegraded': '2024-05-11T01:00:59.669674+0000', 'last_fullsized': '2024-05-11T01:00:59.669674+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.465 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.667366+0000', 'last_change': '2024-05-11T01:00:59.667366+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.667366+0000', 'last_undegraded': '2024-05-11T01:00:59.667366+0000', 'last_fullsized': '2024-05-11T01:00:59.667366+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:PG 2.e is not active+clean 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.669793+0000', 'last_change': '2024-05-11T01:00:59.669793+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.669793+0000', 'last_undegraded': '2024-05-11T01:00:59.669793+0000', 'last_fullsized': '2024-05-11T01:00:59.669793+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666432+0000', 'last_change': '2024-05-11T01:00:59.666432+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666432+0000', 'last_undegraded': '2024-05-11T01:00:59.666432+0000', 'last_fullsized': '2024-05-11T01:00:59.666432+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-05-11T01:01:06.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.669707+0000', 'last_change': '2024-05-11T01:00:59.669707+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.669707+0000', 'last_undegraded': '2024-05-11T01:00:59.669707+0000', 'last_fullsized': '2024-05-11T01:00:59.669707+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666394+0000', 'last_change': '2024-05-11T01:00:59.666394+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666394+0000', 'last_undegraded': '2024-05-11T01:00:59.666394+0000', 'last_fullsized': '2024-05-11T01:00:59.666394+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.663893+0000', 'last_change': '2024-05-11T01:00:59.663893+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.663893+0000', 'last_undegraded': '2024-05-11T01:00:59.663893+0000', 'last_fullsized': '2024-05-11T01:00:59.663893+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-05-11T01:01:06.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.670085+0000', 'last_change': '2024-05-11T01:00:59.670085+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.670085+0000', 'last_undegraded': '2024-05-11T01:00:59.670085+0000', 'last_fullsized': '2024-05-11T01:00:59.670085+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666499+0000', 'last_change': '2024-05-11T01:00:59.666499+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666499+0000', 'last_undegraded': '2024-05-11T01:00:59.666499+0000', 'last_fullsized': '2024-05-11T01:00:59.666499+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666195+0000', 'last_change': '2024-05-11T01:00:59.666195+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666195+0000', 'last_undegraded': '2024-05-11T01:00:59.666195+0000', 'last_fullsized': '2024-05-11T01:00:59.666195+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.468 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.663953+0000', 'last_change': '2024-05-11T01:00:59.663953+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.663953+0000', 'last_undegraded': '2024-05-11T01:00:59.663953+0000', 'last_fullsized': '2024-05-11T01:00:59.663953+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666984+0000', 'last_change': '2024-05-11T01:00:59.666984+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666984+0000', 'last_undegraded': '2024-05-11T01:00:59.666984+0000', 'last_fullsized': '2024-05-11T01:00:59.666984+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-05-11T01:01:06.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.665681+0000', 'last_change': '2024-05-11T01:00:59.665681+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.665681+0000', 'last_undegraded': '2024-05-11T01:00:59.665681+0000', 'last_fullsized': '2024-05-11T01:00:59.665681+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:PG 2.9 is not active+clean 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666285+0000', 'last_change': '2024-05-11T01:00:59.666285+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666285+0000', 'last_undegraded': '2024-05-11T01:00:59.666285+0000', 'last_fullsized': '2024-05-11T01:00:59.666285+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.666055+0000', 'last_change': '2024-05-11T01:00:59.666055+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.666055+0000', 'last_undegraded': '2024-05-11T01:00:59.666055+0000', 'last_fullsized': '2024-05-11T01:00:59.666055+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.470 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.663916+0000', 'last_change': '2024-05-11T01:00:59.663916+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.663916+0000', 'last_undegraded': '2024-05-11T01:00:59.663916+0000', 'last_fullsized': '2024-05-11T01:00:59.663916+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:01:00.671239+0000', 'last_change': '2024-05-11T01:01:00.671239+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.671239+0000', 'last_undegraded': '2024-05-11T01:01:00.671239+0000', 'last_fullsized': '2024-05-11T01:01:00.671239+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.664011+0000', 'last_change': '2024-05-11T01:00:59.664011+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.664011+0000', 'last_undegraded': '2024-05-11T01:00:59.664011+0000', 'last_fullsized': '2024-05-11T01:00:59.664011+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.664671+0000', 'last_change': '2024-05-11T01:00:59.664671+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.664671+0000', 'last_undegraded': '2024-05-11T01:00:59.664671+0000', 'last_fullsized': '2024-05-11T01:00:59.664671+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T01:01:06.471 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-05-11T01:01:06.472 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.472 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-05-11T01:01:06.472 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.663635+0000', 'last_change': '2024-05-11T01:00:59.663635+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.663635+0000', 'last_undegraded': '2024-05-11T01:00:59.663635+0000', 'last_fullsized': '2024-05-11T01:00:59.663635+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T01:01:06.472 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-05-11T01:01:06.472 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-11T01:01:00.661168+0000', 'last_change': '2024-05-11T01:01:00.661168+0000', 'last_active': '2024-05-11T01:01:00.661168+0000', 'last_peered': '2024-05-11T01:01:00.661168+0000', 'last_clean': '2024-05-11T01:01:00.661168+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:01:00.661168+0000', 'last_undegraded': '2024-05-11T01:01:00.661168+0000', 'last_fullsized': '2024-05-11T01:01:00.661168+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'last_clean_scrub_stamp': '2024-05-11T01:01:00.661168+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-11T01:01:06.472 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1f is not active+clean 2024-05-11T01:01:06.472 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-05-11T01:00:59.667046+0000', 'last_change': '2024-05-11T01:00:59.667046+0000', 'last_active': '2024-05-11T01:00:59.657087+0000', 'last_peered': '2024-05-11T01:00:59.657087+0000', 'last_clean': '2024-05-11T01:00:59.657087+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T01:00:59.667046+0000', 'last_undegraded': '2024-05-11T01:00:59.667046+0000', 'last_fullsized': '2024-05-11T01:00:59.667046+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'last_clean_scrub_stamp': '2024-05-11T01:00:59.657087+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T01:01:06.473 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-11T01:01:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.595+0000 7fae068e8640 1 Processor -- start 2024-05-11T01:01:06.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.595+0000 7fae068e8640 1 -- start start 2024-05-11T01:01:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.595+0000 7fae068e8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fae0015f4d0 0x7fae000d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.595+0000 7fae068e8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 0x7fae000d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.595+0000 7fae068e8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0006e710 con 0x7fae000d30f0 2024-05-11T01:01:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.595+0000 7fae068e8640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fae000d2570 con 0x7fae0015f4d0 2024-05-11T01:01:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.595+0000 7fae068e8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fae0015f8b0 con 0x7fae0006dfa0 2024-05-11T01:01:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.596+0000 7fadfffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fae0015f4d0 0x7fae000d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.596+0000 7fadfffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fae0015f4d0 0x7fae000d3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60358/0 (socket says 172.21.15.52:60358) 2024-05-11T01:01:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.596+0000 7fadfffff640 1 -- 172.21.15.52:0/1883892660 learned_addr learned my addr 172.21.15.52:0/1883892660 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.596+0000 7fadff7fe640 1 --2- 172.21.15.52:0/1883892660 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 0x7fae000d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.596+0000 7fadfffff640 1 -- 172.21.15.52:0/1883892660 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 msgr2=0x7fae000d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.596+0000 7fadfffff640 1 --2- 172.21.15.52:0/1883892660 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 0x7fae000d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.596+0000 7fadfffff640 1 -- 172.21.15.52:0/1883892660 >> v1:172.21.15.119:6789/0 conn(0x7fae000d30f0 legacy=0x7fae000ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.596+0000 7fadfffff640 1 -- 172.21.15.52:0/1883892660 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae000d1cd0 con 0x7fae0015f4d0 2024-05-11T01:01:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fadfffff640 1 --2- 172.21.15.52:0/1883892660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae0015f4d0 0x7fae000d3750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadf00098a0 tx=0x7fadf002a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05e9eaa068e30f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fadfeffd640 1 -- 172.21.15.52:0/1883892660 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0037070 con 0x7fae0015f4d0 2024-05-11T01:01:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fadfeffd640 1 -- 172.21.15.52:0/1883892660 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf0032640 con 0x7fae0015f4d0 2024-05-11T01:01:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fadfeffd640 1 -- 172.21.15.52:0/1883892660 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0032a10 con 0x7fae0015f4d0 2024-05-11T01:01:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fae068e8640 1 -- 172.21.15.52:0/1883892660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae0015f4d0 msgr2=0x7fae000d3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fae068e8640 1 --2- 172.21.15.52:0/1883892660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae0015f4d0 0x7fae000d3750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadf00098a0 tx=0x7fadf002a320 comp rx=0 tx=0).stop 2024-05-11T01:01:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fae068e8640 1 -- 172.21.15.52:0/1883892660 shutdown_connections 2024-05-11T01:01:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fae068e8640 1 --2- 172.21.15.52:0/1883892660 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 0x7fae000d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fae068e8640 1 --2- 172.21.15.52:0/1883892660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae0015f4d0 0x7fae000d3750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fae068e8640 1 -- 172.21.15.52:0/1883892660 >> 172.21.15.52:0/1883892660 conn(0x7fae0009ba20 msgr2=0x7fae0009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:06.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fae068e8640 1 -- 172.21.15.52:0/1883892660 shutdown_connections 2024-05-11T01:01:06.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.597+0000 7fae068e8640 1 -- 172.21.15.52:0/1883892660 wait complete. 2024-05-11T01:01:06.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.598+0000 7fae068e8640 1 Processor -- start 2024-05-11T01:01:06.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.599+0000 7fae068e8640 1 -- start start 2024-05-11T01:01:06.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.599+0000 7fae068e8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 0x7fae000b8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.599+0000 7fae068e8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae000d30f0 0x7fae00070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.599+0000 7fae068e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0015f4d0 0x7fae00070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.599+0000 7fae068e8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae0006e3e0 con 0x7fae000d30f0 2024-05-11T01:01:06.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.599+0000 7fae068e8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fae000d2290 con 0x7fae0015f4d0 2024-05-11T01:01:06.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.599+0000 7fae068e8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fae000cede0 con 0x7fae0006dfa0 2024-05-11T01:01:06.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadff7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae000d30f0 0x7fae00070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadff7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae000d30f0 0x7fae00070990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60374/0 (socket says 172.21.15.52:60374) 2024-05-11T01:01:06.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadff7fe640 1 -- 172.21.15.52:0/1812897980 learned_addr learned my addr 172.21.15.52:0/1812897980 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:06.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadfffff640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 0x7fae000b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fae04e85640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0015f4d0 0x7fae00070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadff7fe640 1 -- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 msgr2=0x7fae000b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadff7fe640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 0x7fae000b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadff7fe640 1 -- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0015f4d0 msgr2=0x7fae00070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadff7fe640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0015f4d0 0x7fae00070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadff7fe640 1 -- 172.21.15.52:0/1812897980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0015fe50 con 0x7fae000d30f0 2024-05-11T01:01:06.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fae04e85640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0015f4d0 0x7fae00070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:06.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadff7fe640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae000d30f0 0x7fae00070990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fadf400d830 tx=0x7fadf400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:06.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadfcff9640 1 -- 172.21.15.52:0/1812897980 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf400b6c0 con 0x7fae000d30f0 2024-05-11T01:01:06.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadfcff9640 1 -- 172.21.15.52:0/1812897980 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf4019070 con 0x7fae000d30f0 2024-05-11T01:01:06.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae00160130 con 0x7fae000d30f0 2024-05-11T01:01:06.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fadfcff9640 1 -- 172.21.15.52:0/1812897980 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf400bac0 con 0x7fae000d30f0 2024-05-11T01:01:06.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.600+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae001606f0 con 0x7fae000d30f0 2024-05-11T01:01:06.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.601+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae00160ad0 con 0x7fae000d30f0 2024-05-11T01:01:06.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.602+0000 7fadfcff9640 1 -- 172.21.15.52:0/1812897980 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fadf401e460 con 0x7fae000d30f0 2024-05-11T01:01:06.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.602+0000 7fadfcff9640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fadec0762d0 0x7fadec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.603+0000 7fadfcff9640 1 -- 172.21.15.52:0/1812897980 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fadf4097c30 con 0x7fae000d30f0 2024-05-11T01:01:06.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.603+0000 7fadfffff640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fadec0762d0 0x7fadec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.603+0000 7fadfffff640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fadec0762d0 0x7fadec078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fadf0034000 tx=0x7fadf002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:06.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.607+0000 7fadfcff9640 1 -- 172.21.15.52:0/1812897980 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fae00160ad0 con 0x7fae000d30f0 2024-05-11T01:01:06.801 INFO:tasks.ceph.mds.b.smithi119.stderr:2024-05-11T01:01:06.800+0000 7fded0c78640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T01:01:06.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.871+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fae0015cc40 con 0x7fae000d30f0 2024-05-11T01:01:06.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.871+0000 7fadfcff9640 1 -- 172.21.15.52:0/1812897980 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1106 (secure 0 0 0) 0x7fadf4061590 con 0x7fae000d30f0 2024-05-11T01:01:06.872 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:06.872 INFO:teuthology.orchestra.run.smithi052.stdout:{"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":21,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1715389255,"num_in_osds":8,"osd_in_since":1715389247,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":57},{"state_name":"creating+peering","count":34},{"state_name":"active+clean","count":6}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":241426432,"bytes_avail":772852686848,"bytes_total":773094113280,"unknown_pgs_ratio":0.58762884140014648,"inactive_pgs_ratio":0.3505154550075531},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4276}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-05-11T01:00:45.092255+0000","services":{}},"progress_events":{}} 2024-05-11T01:01:06.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fadec0762d0 msgr2=0x7fadec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fadec0762d0 0x7fadec078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fadf0034000 tx=0x7fadf002e000 comp rx=0 tx=0).stop 2024-05-11T01:01:06.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae000d30f0 msgr2=0x7fae00070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae000d30f0 0x7fae00070990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fadf400d830 tx=0x7fadf400dcf0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 shutdown_connections 2024-05-11T01:01:06.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fadec0762d0 0x7fadec078790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0015f4d0 0x7fae00070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae000d30f0 0x7fae00070990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 --2- 172.21.15.52:0/1812897980 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae0006dfa0 0x7fae000b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 >> 172.21.15.52:0/1812897980 conn(0x7fae0009ba20 msgr2=0x7fae000cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:06.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 shutdown_connections 2024-05-11T01:01:06.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.873+0000 7fae068e8640 1 -- 172.21.15.52:0/1812897980 wait complete. 2024-05-11T01:01:06.885 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-11T01:01:06.885 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:01:06.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.988+0000 7f56f7a93640 1 Processor -- start 2024-05-11T01:01:06.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.988+0000 7f56f7a93640 1 -- start start 2024-05-11T01:01:06.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f7a93640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f56f015f430 0x7f56f015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f7a93640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f00d3d90 0x7f56f00d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f7a93640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f0163250 con 0x7f56f01602b0 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f7a93640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f56f0163530 con 0x7f56f015f430 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f7a93640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f56f0163810 con 0x7f56f00d3d90 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f582f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f56f015f430 0x7f56f015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f502e640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f56f01602b0 0x7f56f00d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44652/0 (socket says 172.21.15.52:44652) 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f502e640 1 -- 172.21.15.52:0/1719312470 learned_addr learned my addr 172.21.15.52:0/1719312470 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f6030640 1 --2- 172.21.15.52:0/1719312470 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f00d3d90 0x7f56f00d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f582f640 1 -- 172.21.15.52:0/1719312470 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f00d3d90 msgr2=0x7f56f00d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f582f640 1 --2- 172.21.15.52:0/1719312470 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f00d3d90 0x7f56f00d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f582f640 1 -- 172.21.15.52:0/1719312470 >> v1:172.21.15.52:6789/0 conn(0x7f56f01602b0 legacy=0x7f56f00d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f582f640 1 -- 172.21.15.52:0/1719312470 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f0163af0 con 0x7f56f015f430 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.989+0000 7f56f482d640 1 -- 172.21.15.52:0/1719312470 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908445163 0 0) 0x7f56f0163250 con 0x7f56f01602b0 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.990+0000 7f56f582f640 1 --2- 172.21.15.52:0/1719312470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 0x7f56f015fd70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009d90 tx=0x7f56e002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d99aabd30cab34b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:06.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.990+0000 7f56f482d640 1 -- 172.21.15.52:0/1719312470 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0007740 con 0x7f56f015f430 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.990+0000 7f56f482d640 1 -- 172.21.15.52:0/1719312470 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e00078e0 con 0x7f56f015f430 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.990+0000 7f56f482d640 1 -- 172.21.15.52:0/1719312470 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0007cb0 con 0x7f56f015f430 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.990+0000 7f56f7a93640 1 -- 172.21.15.52:0/1719312470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 msgr2=0x7f56f015fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.990+0000 7f56f7a93640 1 --2- 172.21.15.52:0/1719312470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 0x7f56f015fd70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009d90 tx=0x7f56e002a660 comp rx=0 tx=0).stop 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.991+0000 7f56f7a93640 1 -- 172.21.15.52:0/1719312470 shutdown_connections 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.991+0000 7f56f7a93640 1 --2- 172.21.15.52:0/1719312470 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f00d3d90 0x7f56f00d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.991+0000 7f56f7a93640 1 --2- 172.21.15.52:0/1719312470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 0x7f56f015fd70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.991+0000 7f56f7a93640 1 -- 172.21.15.52:0/1719312470 >> 172.21.15.52:0/1719312470 conn(0x7f56f008eb70 msgr2=0x7f56f009e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.991+0000 7f56f7a93640 1 -- 172.21.15.52:0/1719312470 shutdown_connections 2024-05-11T01:01:06.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.991+0000 7f56f7a93640 1 -- 172.21.15.52:0/1719312470 wait complete. 2024-05-11T01:01:06.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.992+0000 7f56f7a93640 1 Processor -- start 2024-05-11T01:01:06.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.992+0000 7f56f7a93640 1 -- start start 2024-05-11T01:01:06.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f7a93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f00d3d90 0x7f56f014a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f7a93640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 0x7f56f00c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f7a93640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f01602b0 0x7f56f00ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f7a93640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0163030 con 0x7f56f015f430 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f7a93640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0162eb0 con 0x7f56f00d3d90 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f7a93640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f56f00d27a0 con 0x7f56f01602b0 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f502e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 0x7f56f00c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f502e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 0x7f56f00c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60398/0 (socket says 172.21.15.52:60398) 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f502e640 1 -- 172.21.15.52:0/3251738146 learned_addr learned my addr 172.21.15.52:0/3251738146 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f6030640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f01602b0 0x7f56f00ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f582f640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f00d3d90 0x7f56f014a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f502e640 1 -- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f01602b0 msgr2=0x7f56f00ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f502e640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f01602b0 0x7f56f00ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f502e640 1 -- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f00d3d90 msgr2=0x7f56f014a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f502e640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f00d3d90 0x7f56f014a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f502e640 1 -- 172.21.15.52:0/3251738146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f014b580 con 0x7f56f015f430 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f502e640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 0x7f56f00c8c70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56e400c410 tx=0x7f56e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56ee7fc640 1 -- 172.21.15.52:0/3251738146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e40074e0 con 0x7f56f015f430 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56ee7fc640 1 -- 172.21.15.52:0/3251738146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e40047e0 con 0x7f56f015f430 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f014b860 con 0x7f56f015f430 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56ee7fc640 1 -- 172.21.15.52:0/3251738146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4004bd0 con 0x7f56f015f430 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.993+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f014c100 con 0x7f56f015f430 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.994+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f0069780 con 0x7f56f015f430 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.995+0000 7f56ee7fc640 1 -- 172.21.15.52:0/3251738146 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f56e4011070 con 0x7f56f015f430 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.995+0000 7f56ee7fc640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f56dc0762d0 0x7f56dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.995+0000 7f56ee7fc640 1 -- 172.21.15.52:0/3251738146 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56e4097250 con 0x7f56f015f430 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.996+0000 7f56f582f640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f56dc0762d0 0x7f56dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:06.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:06.996+0000 7f56f582f640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f56dc0762d0 0x7f56dc078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f56e0034000 tx=0x7f56e002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:07.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.000+0000 7f56ee7fc640 1 -- 172.21.15.52:0/3251738146 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f56e4060bb0 con 0x7f56f015f430 2024-05-11T01:01:07.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.186+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56f014c2d0 con 0x7f56dc0762d0 2024-05-11T01:01:07.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.192+0000 7f56ee7fc640 1 -- 172.21.15.52:0/3251738146 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240868 (secure 0 0 0) 0x7f56f014c2d0 con 0x7f56dc0762d0 2024-05-11T01:01:07.196 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:07.197 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:01:07.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f56dc0762d0 msgr2=0x7f56dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:07.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f56dc0762d0 0x7f56dc078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f56e0034000 tx=0x7f56e002f000 comp rx=0 tx=0).stop 2024-05-11T01:01:07.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 msgr2=0x7f56f00c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:07.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 0x7f56f00c8c70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56e400c410 tx=0x7f56e400c8d0 comp rx=0 tx=0).stop 2024-05-11T01:01:07.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 shutdown_connections 2024-05-11T01:01:07.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f56dc0762d0 0x7f56dc078790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:07.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56f01602b0 0x7f56f00ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:07.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56f015f430 0x7f56f00c8c70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:07.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 --2- 172.21.15.52:0/3251738146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f00d3d90 0x7f56f014a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:07.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 >> 172.21.15.52:0/3251738146 conn(0x7f56f008eb70 msgr2=0x7f56f00cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:07.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 shutdown_connections 2024-05-11T01:01:07.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:07.195+0000 7f56f7a93640 1 -- 172.21.15.52:0/3251738146 wait complete. 2024-05-11T01:01:07.207 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-05-11T01:01:05.510926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":2460,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":2519040,"data_stored":1585351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.845534"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672557+0000","last_change":"2024-05-11T01:01:00.672557+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672557+0000","last_undegraded":"2024-05-11T01:01:00.672557+0000","last_fullsized":"2024-05-11T01:01:00.672557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.670954+0000","last_change":"2024-05-11T01:01:00.670954+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.670954+0000","last_undegraded":"2024-05-11T01:01:00.670954+0000","last_fullsized":"2024-05-11T01:01:00.670954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.671724+0000","last_change":"2024-05-11T01:01:00.671724+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.671724+0000","last_undegraded":"2024-05-11T01:01:00.671724+0000","last_fullsized":"2024-05-11T01:01:00.671724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672102+0000","last_change":"2024-05-11T01:01:00.672102+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672102+0000","last_undegraded":"2024-05-11T01:01:00.672102+0000","last_fullsized":"2024-05-11T01:01:00.672102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.672076+0000","last_change":"2024-05-11T01:01:00.672076+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.672076+0000","last_undegraded":"2024-05-11T01:01:00.672076+0000","last_fullsized":"2024-05-11T01:01:00.672076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663676+0000","last_change":"2024-05-11T01:00:59.663676+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663676+0000","last_undegraded":"2024-05-11T01:00:59.663676+0000","last_fullsized":"2024-05-11T01:00:59.663676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685833+0000","last_change":"2024-05-11T01:01:00.684578+0000","last_active":"2024-05-11T01:01:00.685833+0000","last_peered":"2024-05-11T01:01:00.685833+0000","last_clean":"2024-05-11T01:01:00.685833+0000","last_became_active":"2024-05-11T01:01:00.684303+0000","last_became_peered":"2024-05-11T01:01:00.684303+0000","last_unstale":"2024-05-11T01:01:00.685833+0000","last_undegraded":"2024-05-11T01:01:00.685833+0000","last_fullsized":"2024-05-11T01:01:00.685833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:24:23.590323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663629+0000","last_change":"2024-05-11T01:00:59.663629+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663629+0000","last_undegraded":"2024-05-11T01:00:59.663629+0000","last_fullsized":"2024-05-11T01:00:59.663629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666535+0000","last_change":"2024-05-11T01:00:59.666535+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666535+0000","last_undegraded":"2024-05-11T01:00:59.666535+0000","last_fullsized":"2024-05-11T01:00:59.666535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.683064+0000","last_change":"2024-05-11T01:01:00.679681+0000","last_active":"2024-05-11T01:01:00.683064+0000","last_peered":"2024-05-11T01:01:00.683064+0000","last_clean":"2024-05-11T01:01:00.683064+0000","last_became_active":"2024-05-11T01:01:00.679125+0000","last_became_peered":"2024-05-11T01:01:00.679125+0000","last_unstale":"2024-05-11T01:01:00.683064+0000","last_undegraded":"2024-05-11T01:01:00.683064+0000","last_fullsized":"2024-05-11T01:01:00.683064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:13.659371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.668251+0000","last_change":"2024-05-11T01:00:59.668251+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.668251+0000","last_undegraded":"2024-05-11T01:00:59.668251+0000","last_fullsized":"2024-05-11T01:00:59.668251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666997+0000","last_change":"2024-05-11T01:00:59.666997+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666997+0000","last_undegraded":"2024-05-11T01:00:59.666997+0000","last_fullsized":"2024-05-11T01:00:59.666997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685683+0000","last_change":"2024-05-11T01:01:00.683575+0000","last_active":"2024-05-11T01:01:00.685683+0000","last_peered":"2024-05-11T01:01:00.685683+0000","last_clean":"2024-05-11T01:01:00.685683+0000","last_became_active":"2024-05-11T01:01:00.683259+0000","last_became_peered":"2024-05-11T01:01:00.683259+0000","last_unstale":"2024-05-11T01:01:00.685683+0000","last_undegraded":"2024-05-11T01:01:00.685683+0000","last_fullsized":"2024-05-11T01:01:00.685683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:38:37.156429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666516+0000","last_change":"2024-05-11T01:00:59.666516+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666516+0000","last_undegraded":"2024-05-11T01:00:59.666516+0000","last_fullsized":"2024-05-11T01:00:59.666516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.668583+0000","last_change":"2024-05-11T01:00:59.668583+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.668583+0000","last_undegraded":"2024-05-11T01:00:59.668583+0000","last_fullsized":"2024-05-11T01:00:59.668583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.685870+0000","last_change":"2024-05-11T01:01:00.684714+0000","last_active":"2024-05-11T01:01:00.685870+0000","last_peered":"2024-05-11T01:01:00.685870+0000","last_clean":"2024-05-11T01:01:00.685870+0000","last_became_active":"2024-05-11T01:01:00.684454+0000","last_became_peered":"2024-05-11T01:01:00.684454+0000","last_unstale":"2024-05-11T01:01:00.685870+0000","last_undegraded":"2024-05-11T01:01:00.685870+0000","last_fullsized":"2024-05-11T01:01:00.685870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:12:31.764619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.670497+0000","last_change":"2024-05-11T01:01:00.670497+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.670497+0000","last_undegraded":"2024-05-11T01:01:00.670497+0000","last_fullsized":"2024-05-11T01:01:00.670497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669674+0000","last_change":"2024-05-11T01:00:59.669674+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669674+0000","last_undegraded":"2024-05-11T01:00:59.669674+0000","last_fullsized":"2024-05-11T01:00:59.669674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.667366+0000","last_change":"2024-05-11T01:00:59.667366+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.667366+0000","last_undegraded":"2024-05-11T01:00:59.667366+0000","last_fullsized":"2024-05-11T01:00:59.667366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669793+0000","last_change":"2024-05-11T01:00:59.669793+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669793+0000","last_undegraded":"2024-05-11T01:00:59.669793+0000","last_fullsized":"2024-05-11T01:00:59.669793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666432+0000","last_change":"2024-05-11T01:00:59.666432+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666432+0000","last_undegraded":"2024-05-11T01:00:59.666432+0000","last_fullsized":"2024-05-11T01:00:59.666432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.669707+0000","last_change":"2024-05-11T01:00:59.669707+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.669707+0000","last_undegraded":"2024-05-11T01:00:59.669707+0000","last_fullsized":"2024-05-11T01:00:59.669707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":104,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T01:00:59.668655+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:00:59.668655+0000","last_peered":"2024-05-11T01:00:59.668655+0000","last_clean":"2024-05-11T01:00:59.668655+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:00:59.668655+0000","last_undegraded":"2024-05-11T01:00:59.668655+0000","last_fullsized":"2024-05-11T01:00:59.668655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666394+0000","last_change":"2024-05-11T01:00:59.666394+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666394+0000","last_undegraded":"2024-05-11T01:00:59.666394+0000","last_fullsized":"2024-05-11T01:00:59.666394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663893+0000","last_change":"2024-05-11T01:00:59.663893+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663893+0000","last_undegraded":"2024-05-11T01:00:59.663893+0000","last_fullsized":"2024-05-11T01:00:59.663893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.670085+0000","last_change":"2024-05-11T01:00:59.670085+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.670085+0000","last_undegraded":"2024-05-11T01:00:59.670085+0000","last_fullsized":"2024-05-11T01:00:59.670085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666499+0000","last_change":"2024-05-11T01:00:59.666499+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666499+0000","last_undegraded":"2024-05-11T01:00:59.666499+0000","last_fullsized":"2024-05-11T01:00:59.666499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666195+0000","last_change":"2024-05-11T01:00:59.666195+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666195+0000","last_undegraded":"2024-05-11T01:00:59.666195+0000","last_fullsized":"2024-05-11T01:00:59.666195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663953+0000","last_change":"2024-05-11T01:00:59.663953+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663953+0000","last_undegraded":"2024-05-11T01:00:59.663953+0000","last_fullsized":"2024-05-11T01:00:59.663953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666984+0000","last_change":"2024-05-11T01:00:59.666984+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666984+0000","last_undegraded":"2024-05-11T01:00:59.666984+0000","last_fullsized":"2024-05-11T01:00:59.666984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T01:01:00.687089+0000","last_change":"2024-05-11T01:01:00.686048+0000","last_active":"2024-05-11T01:01:00.687089+0000","last_peered":"2024-05-11T01:01:00.687089+0000","last_clean":"2024-05-11T01:01:00.687089+0000","last_became_active":"2024-05-11T01:01:00.685215+0000","last_became_peered":"2024-05-11T01:01:00.685215+0000","last_unstale":"2024-05-11T01:01:00.687089+0000","last_undegraded":"2024-05-11T01:01:00.687089+0000","last_fullsized":"2024-05-11T01:01:00.687089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:00:17.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.665681+0000","last_change":"2024-05-11T01:00:59.665681+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.665681+0000","last_undegraded":"2024-05-11T01:00:59.665681+0000","last_fullsized":"2024-05-11T01:00:59.665681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666285+0000","last_change":"2024-05-11T01:00:59.666285+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666285+0000","last_undegraded":"2024-05-11T01:00:59.666285+0000","last_fullsized":"2024-05-11T01:00:59.666285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.666055+0000","last_change":"2024-05-11T01:00:59.666055+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.666055+0000","last_undegraded":"2024-05-11T01:00:59.666055+0000","last_fullsized":"2024-05-11T01:00:59.666055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663916+0000","last_change":"2024-05-11T01:00:59.663916+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663916+0000","last_undegraded":"2024-05-11T01:00:59.663916+0000","last_fullsized":"2024-05-11T01:00:59.663916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-05-11T01:01:00.671239+0000","last_change":"2024-05-11T01:01:00.671239+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.671239+0000","last_undegraded":"2024-05-11T01:01:00.671239+0000","last_fullsized":"2024-05-11T01:01:00.671239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.664011+0000","last_change":"2024-05-11T01:00:59.664011+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.664011+0000","last_undegraded":"2024-05-11T01:00:59.664011+0000","last_fullsized":"2024-05-11T01:00:59.664011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.664671+0000","last_change":"2024-05-11T01:00:59.664671+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.664671+0000","last_undegraded":"2024-05-11T01:00:59.664671+0000","last_fullsized":"2024-05-11T01:00:59.664671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.663635+0000","last_change":"2024-05-11T01:00:59.663635+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.663635+0000","last_undegraded":"2024-05-11T01:00:59.663635+0000","last_fullsized":"2024-05-11T01:00:59.663635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:01:00.661168+0000","last_change":"2024-05-11T01:01:00.661168+0000","last_active":"2024-05-11T01:01:00.661168+0000","last_peered":"2024-05-11T01:01:00.661168+0000","last_clean":"2024-05-11T01:01:00.661168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:01:00.661168+0000","last_undegraded":"2024-05-11T01:01:00.661168+0000","last_fullsized":"2024-05-11T01:01:00.661168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-11T01:00:59.667046+0000","last_change":"2024-05-11T01:00:59.667046+0000","last_active":"2024-05-11T01:00:59.657087+0000","last_peered":"2024-05-11T01:00:59.657087+0000","last_clean":"2024-05-11T01:00:59.657087+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:00:59.667046+0000","last_undegraded":"2024-05-11T01:00:59.667046+0000","last_fullsized":"2024-05-11T01:00:59.667046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":761856,"data_stored":641672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":761856,"data_stored":641672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:01:10.211 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:01:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.327+0000 7f97cc6a7640 1 Processor -- start 2024-05-11T01:01:10.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.327+0000 7f97cc6a7640 1 -- start start 2024-05-11T01:01:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cc6a7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97c40d2ea0 0x7f97c40d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cc6a7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f97c40b2600 0x7f97c40b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cc6a7640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c40d1f90 con 0x7f97c415f2d0 2024-05-11T01:01:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cc6a7640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f97c40d2270 con 0x7f97c40b2600 2024-05-11T01:01:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cc6a7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f97c40d2550 con 0x7f97c40d2ea0 2024-05-11T01:01:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cac44640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f97c40b2600 0x7f97c40b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97ca443640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f97c415f2d0 0x7f97c406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44662/0 (socket says 172.21.15.52:44662) 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97ca443640 1 -- 172.21.15.52:0/158120840 learned_addr learned my addr 172.21.15.52:0/158120840 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97c9c42640 1 --2- 172.21.15.52:0/158120840 >> v2:172.21.15.119:3300/0 conn(0x7f97c40d2ea0 0x7f97c40d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cac44640 1 -- 172.21.15.52:0/158120840 >> v2:172.21.15.119:3300/0 conn(0x7f97c40d2ea0 msgr2=0x7f97c40d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cac44640 1 --2- 172.21.15.52:0/158120840 >> v2:172.21.15.119:3300/0 conn(0x7f97c40d2ea0 0x7f97c40d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cac44640 1 -- 172.21.15.52:0/158120840 >> v1:172.21.15.52:6789/0 conn(0x7f97c415f2d0 legacy=0x7f97c406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97c9441640 1 -- 172.21.15.52:0/158120840 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2695335107 0 0) 0x7f97c40d1f90 con 0x7f97c415f2d0 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.328+0000 7f97cac44640 1 -- 172.21.15.52:0/158120840 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c40d2830 con 0x7f97c40b2600 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.329+0000 7f97cac44640 1 --2- 172.21.15.52:0/158120840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40b2600 0x7f97c40b2a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f97c40c9f90 tx=0x7f97bc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e6dd62d6070968 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.329+0000 7f97c9441640 1 -- 172.21.15.52:0/158120840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc007740 con 0x7f97c40b2600 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.329+0000 7f97c9441640 1 -- 172.21.15.52:0/158120840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97bc0078e0 con 0x7f97c40b2600 2024-05-11T01:01:10.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.329+0000 7f97c9441640 1 -- 172.21.15.52:0/158120840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc007cb0 con 0x7f97c40b2600 2024-05-11T01:01:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.330+0000 7f97cc6a7640 1 -- 172.21.15.52:0/158120840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40b2600 msgr2=0x7f97c40b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.330+0000 7f97cc6a7640 1 --2- 172.21.15.52:0/158120840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40b2600 0x7f97c40b2a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f97c40c9f90 tx=0x7f97bc02a680 comp rx=0 tx=0).stop 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.330+0000 7f97cc6a7640 1 -- 172.21.15.52:0/158120840 shutdown_connections 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.330+0000 7f97cc6a7640 1 --2- 172.21.15.52:0/158120840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40b2600 0x7f97c40b2a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.330+0000 7f97cc6a7640 1 --2- 172.21.15.52:0/158120840 >> v2:172.21.15.119:3300/0 conn(0x7f97c40d2ea0 0x7f97c40d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.330+0000 7f97cc6a7640 1 -- 172.21.15.52:0/158120840 >> 172.21.15.52:0/158120840 conn(0x7f97c409ba20 msgr2=0x7f97c409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.330+0000 7f97cc6a7640 1 -- 172.21.15.52:0/158120840 shutdown_connections 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.330+0000 7f97cc6a7640 1 -- 172.21.15.52:0/158120840 wait complete. 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.331+0000 7f97cc6a7640 1 Processor -- start 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.331+0000 7f97cc6a7640 1 -- start start 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97cc6a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97c40b2600 0x7f97c41624f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97cc6a7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40d2ea0 0x7f97c41493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97cc6a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97c415f2d0 0x7f97c41561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97cc6a7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97c415f830 con 0x7f97c40d2ea0 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97cc6a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97c415f6b0 con 0x7f97c415f2d0 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97cc6a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f97c40d1b80 con 0x7f97c40b2600 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97c9c42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40d2ea0 0x7f97c41493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97c9c42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40d2ea0 0x7f97c41493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60416/0 (socket says 172.21.15.52:60416) 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97c9c42640 1 -- 172.21.15.52:0/1450571062 learned_addr learned my addr 172.21.15.52:0/1450571062 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97ca443640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97c40b2600 0x7f97c41624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97cac44640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97c415f2d0 0x7f97c41561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97c9c42640 1 -- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97c40b2600 msgr2=0x7f97c41624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97c9c42640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97c40b2600 0x7f97c41624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97c9c42640 1 -- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97c415f2d0 msgr2=0x7f97c41561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97c9c42640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97c415f2d0 0x7f97c41561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97c9c42640 1 -- 172.21.15.52:0/1450571062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c4154e20 con 0x7f97c40d2ea0 2024-05-11T01:01:10.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.332+0000 7f97c9c42640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40d2ea0 0x7f97c41493c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97b400d800 tx=0x7f97b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.333+0000 7f97baffd640 1 -- 172.21.15.52:0/1450571062 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b400b6c0 con 0x7f97c40d2ea0 2024-05-11T01:01:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.333+0000 7f97baffd640 1 -- 172.21.15.52:0/1450571062 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b4019070 con 0x7f97c40d2ea0 2024-05-11T01:01:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.333+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c4155100 con 0x7f97c40d2ea0 2024-05-11T01:01:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.333+0000 7f97baffd640 1 -- 172.21.15.52:0/1450571062 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b400ba80 con 0x7f97c40d2ea0 2024-05-11T01:01:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.333+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c41556c0 con 0x7f97c40d2ea0 2024-05-11T01:01:10.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.334+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c4069780 con 0x7f97c40d2ea0 2024-05-11T01:01:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.334+0000 7f97baffd640 1 -- 172.21.15.52:0/1450571062 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f97b400bc20 con 0x7f97c40d2ea0 2024-05-11T01:01:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.335+0000 7f97baffd640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f97a8076300 0x7f97a80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.335+0000 7f97baffd640 1 -- 172.21.15.52:0/1450571062 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f97b40979a0 con 0x7f97c40d2ea0 2024-05-11T01:01:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.335+0000 7f97ca443640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f97a8076300 0x7f97a80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.336+0000 7f97ca443640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f97a8076300 0x7f97a80787c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97c0007fd0 tx=0x7f97c0007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:10.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.339+0000 7f97baffd640 1 -- 172.21.15.52:0/1450571062 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f97b4061390 con 0x7f97c40d2ea0 2024-05-11T01:01:10.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.528+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97c415e760 con 0x7f97a8076300 2024-05-11T01:01:10.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.534+0000 7f97baffd640 1 -- 172.21.15.52:0/1450571062 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7f97c415e760 con 0x7f97a8076300 2024-05-11T01:01:10.539 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:10.540 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:01:10.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f97a8076300 msgr2=0x7f97a80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f97a8076300 0x7f97a80787c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97c0007fd0 tx=0x7f97c0007de0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40d2ea0 msgr2=0x7f97c41493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40d2ea0 0x7f97c41493c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97b400d800 tx=0x7f97b400dcc0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 shutdown_connections 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f97a8076300 0x7f97a80787c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97c415f2d0 0x7f97c41561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c40d2ea0 0x7f97c41493c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 --2- 172.21.15.52:0/1450571062 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97c40b2600 0x7f97c41624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.537+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 >> 172.21.15.52:0/1450571062 conn(0x7f97c409ba20 msgr2=0x7f97c40afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.538+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 shutdown_connections 2024-05-11T01:01:10.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.538+0000 7f97cc6a7640 1 -- 172.21.15.52:0/1450571062 wait complete. 2024-05-11T01:01:10.549 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-11T01:01:09.512002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235632,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739088,"statfs":{"total":773094113280,"available":772852826112,"internally_reserved":0,"allocated":2891776,"data_stored":1750788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001924"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794175+0000","last_change":"2024-05-11T01:01:01.690368+0000","last_active":"2024-05-11T01:01:02.794175+0000","last_peered":"2024-05-11T01:01:02.794175+0000","last_clean":"2024-05-11T01:01:02.794175+0000","last_became_active":"2024-05-11T01:01:01.689860+0000","last_became_peered":"2024-05-11T01:01:01.689860+0000","last_unstale":"2024-05-11T01:01:02.794175+0000","last_undegraded":"2024-05-11T01:01:02.794175+0000","last_fullsized":"2024-05-11T01:01:02.794175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:22:27.780116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703164+0000","last_change":"2024-05-11T01:01:01.692121+0000","last_active":"2024-05-11T01:01:02.703164+0000","last_peered":"2024-05-11T01:01:02.703164+0000","last_clean":"2024-05-11T01:01:02.703164+0000","last_became_active":"2024-05-11T01:01:01.691843+0000","last_became_peered":"2024-05-11T01:01:01.691843+0000","last_unstale":"2024-05-11T01:01:02.703164+0000","last_undegraded":"2024-05-11T01:01:02.703164+0000","last_fullsized":"2024-05-11T01:01:02.703164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:04:19.015364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676930+0000","last_change":"2024-05-11T01:01:01.681630+0000","last_active":"2024-05-11T01:01:02.676930+0000","last_peered":"2024-05-11T01:01:02.676930+0000","last_clean":"2024-05-11T01:01:02.676930+0000","last_became_active":"2024-05-11T01:01:01.681342+0000","last_became_peered":"2024-05-11T01:01:01.681342+0000","last_unstale":"2024-05-11T01:01:02.676930+0000","last_undegraded":"2024-05-11T01:01:02.676930+0000","last_fullsized":"2024-05-11T01:01:02.676930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:02:55.828068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678747+0000","last_change":"2024-05-11T01:01:01.687729+0000","last_active":"2024-05-11T01:01:02.678747+0000","last_peered":"2024-05-11T01:01:02.678747+0000","last_clean":"2024-05-11T01:01:02.678747+0000","last_became_active":"2024-05-11T01:01:01.687234+0000","last_became_peered":"2024-05-11T01:01:01.687234+0000","last_unstale":"2024-05-11T01:01:02.678747+0000","last_undegraded":"2024-05-11T01:01:02.678747+0000","last_fullsized":"2024-05-11T01:01:02.678747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:10.357158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793604+0000","last_change":"2024-05-11T01:01:01.687598+0000","last_active":"2024-05-11T01:01:02.793604+0000","last_peered":"2024-05-11T01:01:02.793604+0000","last_clean":"2024-05-11T01:01:02.793604+0000","last_became_active":"2024-05-11T01:01:01.687326+0000","last_became_peered":"2024-05-11T01:01:01.687326+0000","last_unstale":"2024-05-11T01:01:02.793604+0000","last_undegraded":"2024-05-11T01:01:02.793604+0000","last_fullsized":"2024-05-11T01:01:02.793604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677883+0000","last_change":"2024-05-11T01:01:01.688617+0000","last_active":"2024-05-11T01:01:02.677883+0000","last_peered":"2024-05-11T01:01:02.677883+0000","last_clean":"2024-05-11T01:01:02.677883+0000","last_became_active":"2024-05-11T01:01:01.687985+0000","last_became_peered":"2024-05-11T01:01:01.687985+0000","last_unstale":"2024-05-11T01:01:02.677883+0000","last_undegraded":"2024-05-11T01:01:02.677883+0000","last_fullsized":"2024-05-11T01:01:02.677883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:24.445230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793490+0000","last_change":"2024-05-11T01:01:01.692158+0000","last_active":"2024-05-11T01:01:02.793490+0000","last_peered":"2024-05-11T01:01:02.793490+0000","last_clean":"2024-05-11T01:01:02.793490+0000","last_became_active":"2024-05-11T01:01:01.691929+0000","last_became_peered":"2024-05-11T01:01:01.691929+0000","last_unstale":"2024-05-11T01:01:02.793490+0000","last_undegraded":"2024-05-11T01:01:02.793490+0000","last_fullsized":"2024-05-11T01:01:02.793490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:24.858093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912954+0000","last_change":"2024-05-11T01:01:01.694128+0000","last_active":"2024-05-11T01:01:02.912954+0000","last_peered":"2024-05-11T01:01:02.912954+0000","last_clean":"2024-05-11T01:01:02.912954+0000","last_became_active":"2024-05-11T01:01:01.693513+0000","last_became_peered":"2024-05-11T01:01:01.693513+0000","last_unstale":"2024-05-11T01:01:02.912954+0000","last_undegraded":"2024-05-11T01:01:02.912954+0000","last_fullsized":"2024-05-11T01:01:02.912954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:02:55.828068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794149+0000","last_change":"2024-05-11T01:01:01.705504+0000","last_active":"2024-05-11T01:01:02.794149+0000","last_peered":"2024-05-11T01:01:02.794149+0000","last_clean":"2024-05-11T01:01:02.794149+0000","last_became_active":"2024-05-11T01:01:01.705258+0000","last_became_peered":"2024-05-11T01:01:01.705258+0000","last_unstale":"2024-05-11T01:01:02.794149+0000","last_undegraded":"2024-05-11T01:01:02.794149+0000","last_fullsized":"2024-05-11T01:01:02.794149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:00:03.997273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705744+0000","last_change":"2024-05-11T01:01:01.696662+0000","last_active":"2024-05-11T01:01:02.705744+0000","last_peered":"2024-05-11T01:01:02.705744+0000","last_clean":"2024-05-11T01:01:02.705744+0000","last_became_active":"2024-05-11T01:01:01.696364+0000","last_became_peered":"2024-05-11T01:01:01.696364+0000","last_unstale":"2024-05-11T01:01:02.705744+0000","last_undegraded":"2024-05-11T01:01:02.705744+0000","last_fullsized":"2024-05-11T01:01:02.705744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:04:19.015364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705571+0000","last_change":"2024-05-11T01:01:01.699790+0000","last_active":"2024-05-11T01:01:02.705571+0000","last_peered":"2024-05-11T01:01:02.705571+0000","last_clean":"2024-05-11T01:01:02.705571+0000","last_became_active":"2024-05-11T01:01:01.699494+0000","last_became_peered":"2024-05-11T01:01:01.699494+0000","last_unstale":"2024-05-11T01:01:02.705571+0000","last_undegraded":"2024-05-11T01:01:02.705571+0000","last_fullsized":"2024-05-11T01:01:02.705571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678303+0000","last_change":"2024-05-11T01:01:01.696604+0000","last_active":"2024-05-11T01:01:02.678303+0000","last_peered":"2024-05-11T01:01:02.678303+0000","last_clean":"2024-05-11T01:01:02.678303+0000","last_became_active":"2024-05-11T01:01:01.695857+0000","last_became_peered":"2024-05-11T01:01:01.695857+0000","last_unstale":"2024-05-11T01:01:02.678303+0000","last_undegraded":"2024-05-11T01:01:02.678303+0000","last_fullsized":"2024-05-11T01:01:02.678303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:06.977149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913350+0000","last_change":"2024-05-11T01:01:01.690950+0000","last_active":"2024-05-11T01:01:02.913350+0000","last_peered":"2024-05-11T01:01:02.913350+0000","last_clean":"2024-05-11T01:01:02.913350+0000","last_became_active":"2024-05-11T01:01:01.690683+0000","last_became_peered":"2024-05-11T01:01:01.690683+0000","last_unstale":"2024-05-11T01:01:02.913350+0000","last_undegraded":"2024-05-11T01:01:02.913350+0000","last_fullsized":"2024-05-11T01:01:02.913350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676647+0000","last_change":"2024-05-11T01:01:01.690525+0000","last_active":"2024-05-11T01:01:02.676647+0000","last_peered":"2024-05-11T01:01:02.676647+0000","last_clean":"2024-05-11T01:01:02.676647+0000","last_became_active":"2024-05-11T01:01:01.690246+0000","last_became_peered":"2024-05-11T01:01:01.690246+0000","last_unstale":"2024-05-11T01:01:02.676647+0000","last_undegraded":"2024-05-11T01:01:02.676647+0000","last_fullsized":"2024-05-11T01:01:02.676647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:59.768543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.702977+0000","last_change":"2024-05-11T01:01:01.696660+0000","last_active":"2024-05-11T01:01:02.702977+0000","last_peered":"2024-05-11T01:01:02.702977+0000","last_clean":"2024-05-11T01:01:02.702977+0000","last_became_active":"2024-05-11T01:01:01.696344+0000","last_became_peered":"2024-05-11T01:01:01.696344+0000","last_unstale":"2024-05-11T01:01:02.702977+0000","last_undegraded":"2024-05-11T01:01:02.702977+0000","last_fullsized":"2024-05-11T01:01:02.702977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677628+0000","last_change":"2024-05-11T01:01:01.691118+0000","last_active":"2024-05-11T01:01:02.677628+0000","last_peered":"2024-05-11T01:01:02.677628+0000","last_clean":"2024-05-11T01:01:02.677628+0000","last_became_active":"2024-05-11T01:01:01.690850+0000","last_became_peered":"2024-05-11T01:01:01.690850+0000","last_unstale":"2024-05-11T01:01:02.677628+0000","last_undegraded":"2024-05-11T01:01:02.677628+0000","last_fullsized":"2024-05-11T01:01:02.677628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678640+0000","last_change":"2024-05-11T01:01:01.691696+0000","last_active":"2024-05-11T01:01:02.678640+0000","last_peered":"2024-05-11T01:01:02.678640+0000","last_clean":"2024-05-11T01:01:02.678640+0000","last_became_active":"2024-05-11T01:01:01.691424+0000","last_became_peered":"2024-05-11T01:01:01.691424+0000","last_unstale":"2024-05-11T01:01:02.678640+0000","last_undegraded":"2024-05-11T01:01:02.678640+0000","last_fullsized":"2024-05-11T01:01:02.678640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:13:49.063574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913015+0000","last_change":"2024-05-11T01:01:01.690778+0000","last_active":"2024-05-11T01:01:02.913015+0000","last_peered":"2024-05-11T01:01:02.913015+0000","last_clean":"2024-05-11T01:01:02.913015+0000","last_became_active":"2024-05-11T01:01:01.690403+0000","last_became_peered":"2024-05-11T01:01:01.690403+0000","last_unstale":"2024-05-11T01:01:02.913015+0000","last_undegraded":"2024-05-11T01:01:02.913015+0000","last_fullsized":"2024-05-11T01:01:02.913015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:59.768543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794101+0000","last_change":"2024-05-11T01:01:01.697511+0000","last_active":"2024-05-11T01:01:02.794101+0000","last_peered":"2024-05-11T01:01:02.794101+0000","last_clean":"2024-05-11T01:01:02.794101+0000","last_became_active":"2024-05-11T01:01:01.697263+0000","last_became_peered":"2024-05-11T01:01:01.697263+0000","last_unstale":"2024-05-11T01:01:02.794101+0000","last_undegraded":"2024-05-11T01:01:02.794101+0000","last_fullsized":"2024-05-11T01:01:02.794101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:10.357158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705536+0000","last_change":"2024-05-11T01:01:01.697194+0000","last_active":"2024-05-11T01:01:02.705536+0000","last_peered":"2024-05-11T01:01:02.705536+0000","last_clean":"2024-05-11T01:01:02.705536+0000","last_became_active":"2024-05-11T01:01:01.696909+0000","last_became_peered":"2024-05-11T01:01:01.696909+0000","last_unstale":"2024-05-11T01:01:02.705536+0000","last_undegraded":"2024-05-11T01:01:02.705536+0000","last_fullsized":"2024-05-11T01:01:02.705536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676985+0000","last_change":"2024-05-11T01:01:01.690623+0000","last_active":"2024-05-11T01:01:02.676985+0000","last_peered":"2024-05-11T01:01:02.676985+0000","last_clean":"2024-05-11T01:01:02.676985+0000","last_became_active":"2024-05-11T01:01:01.689762+0000","last_became_peered":"2024-05-11T01:01:01.689762+0000","last_unstale":"2024-05-11T01:01:02.676985+0000","last_undegraded":"2024-05-11T01:01:02.676985+0000","last_fullsized":"2024-05-11T01:01:02.676985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:10:39.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678044+0000","last_change":"2024-05-11T01:01:01.692409+0000","last_active":"2024-05-11T01:01:02.678044+0000","last_peered":"2024-05-11T01:01:02.678044+0000","last_clean":"2024-05-11T01:01:02.678044+0000","last_became_active":"2024-05-11T01:01:01.692117+0000","last_became_peered":"2024-05-11T01:01:01.692117+0000","last_unstale":"2024-05-11T01:01:02.678044+0000","last_undegraded":"2024-05-11T01:01:02.678044+0000","last_fullsized":"2024-05-11T01:01:02.678044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:36:33.062657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913338+0000","last_change":"2024-05-11T01:01:01.697812+0000","last_active":"2024-05-11T01:01:02.913338+0000","last_peered":"2024-05-11T01:01:02.913338+0000","last_clean":"2024-05-11T01:01:02.913338+0000","last_became_active":"2024-05-11T01:01:01.697509+0000","last_became_peered":"2024-05-11T01:01:01.697509+0000","last_unstale":"2024-05-11T01:01:02.913338+0000","last_undegraded":"2024-05-11T01:01:02.913338+0000","last_fullsized":"2024-05-11T01:01:02.913338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678573+0000","last_change":"2024-05-11T01:01:01.689599+0000","last_active":"2024-05-11T01:01:02.678573+0000","last_peered":"2024-05-11T01:01:02.678573+0000","last_clean":"2024-05-11T01:01:02.678573+0000","last_became_active":"2024-05-11T01:01:01.686994+0000","last_became_peered":"2024-05-11T01:01:01.686994+0000","last_unstale":"2024-05-11T01:01:02.678573+0000","last_undegraded":"2024-05-11T01:01:02.678573+0000","last_fullsized":"2024-05-11T01:01:02.678573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:32.857796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913719+0000","last_change":"2024-05-11T01:01:01.699000+0000","last_active":"2024-05-11T01:01:02.913719+0000","last_peered":"2024-05-11T01:01:02.913719+0000","last_clean":"2024-05-11T01:01:02.913719+0000","last_became_active":"2024-05-11T01:01:01.698640+0000","last_became_peered":"2024-05-11T01:01:01.698640+0000","last_unstale":"2024-05-11T01:01:02.913719+0000","last_undegraded":"2024-05-11T01:01:02.913719+0000","last_fullsized":"2024-05-11T01:01:02.913719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:24.445230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705756+0000","last_change":"2024-05-11T01:01:01.689683+0000","last_active":"2024-05-11T01:01:02.705756+0000","last_peered":"2024-05-11T01:01:02.705756+0000","last_clean":"2024-05-11T01:01:02.705756+0000","last_became_active":"2024-05-11T01:01:01.689306+0000","last_became_peered":"2024-05-11T01:01:01.689306+0000","last_unstale":"2024-05-11T01:01:02.705756+0000","last_undegraded":"2024-05-11T01:01:02.705756+0000","last_fullsized":"2024-05-11T01:01:02.705756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703176+0000","last_change":"2024-05-11T01:01:01.694512+0000","last_active":"2024-05-11T01:01:02.703176+0000","last_peered":"2024-05-11T01:01:02.703176+0000","last_clean":"2024-05-11T01:01:02.703176+0000","last_became_active":"2024-05-11T01:01:01.694214+0000","last_became_peered":"2024-05-11T01:01:01.694214+0000","last_unstale":"2024-05-11T01:01:02.703176+0000","last_undegraded":"2024-05-11T01:01:02.703176+0000","last_fullsized":"2024-05-11T01:01:02.703176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913695+0000","last_change":"2024-05-11T01:01:01.698983+0000","last_active":"2024-05-11T01:01:02.913695+0000","last_peered":"2024-05-11T01:01:02.913695+0000","last_clean":"2024-05-11T01:01:02.913695+0000","last_became_active":"2024-05-11T01:01:01.698682+0000","last_became_peered":"2024-05-11T01:01:01.698682+0000","last_unstale":"2024-05-11T01:01:02.913695+0000","last_undegraded":"2024-05-11T01:01:02.913695+0000","last_fullsized":"2024-05-11T01:01:02.913695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:04:19.015364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676868+0000","last_change":"2024-05-11T01:01:01.690633+0000","last_active":"2024-05-11T01:01:02.676868+0000","last_peered":"2024-05-11T01:01:02.676868+0000","last_clean":"2024-05-11T01:01:02.676868+0000","last_became_active":"2024-05-11T01:01:01.690227+0000","last_became_peered":"2024-05-11T01:01:01.690227+0000","last_unstale":"2024-05-11T01:01:02.676868+0000","last_undegraded":"2024-05-11T01:01:02.676868+0000","last_fullsized":"2024-05-11T01:01:02.676868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:24.703345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678131+0000","last_change":"2024-05-11T01:01:01.688827+0000","last_active":"2024-05-11T01:01:02.678131+0000","last_peered":"2024-05-11T01:01:02.678131+0000","last_clean":"2024-05-11T01:01:02.678131+0000","last_became_active":"2024-05-11T01:01:01.688376+0000","last_became_peered":"2024-05-11T01:01:01.688376+0000","last_unstale":"2024-05-11T01:01:02.678131+0000","last_undegraded":"2024-05-11T01:01:02.678131+0000","last_fullsized":"2024-05-11T01:01:02.678131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:24.858093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912912+0000","last_change":"2024-05-11T01:01:01.692633+0000","last_active":"2024-05-11T01:01:02.912912+0000","last_peered":"2024-05-11T01:01:02.912912+0000","last_clean":"2024-05-11T01:01:02.912912+0000","last_became_active":"2024-05-11T01:01:01.692305+0000","last_became_peered":"2024-05-11T01:01:01.692305+0000","last_unstale":"2024-05-11T01:01:02.912912+0000","last_undegraded":"2024-05-11T01:01:02.912912+0000","last_fullsized":"2024-05-11T01:01:02.912912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:10:34.198396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705339+0000","last_change":"2024-05-11T01:01:01.692423+0000","last_active":"2024-05-11T01:01:02.705339+0000","last_peered":"2024-05-11T01:01:02.705339+0000","last_clean":"2024-05-11T01:01:02.705339+0000","last_became_active":"2024-05-11T01:01:01.691669+0000","last_became_peered":"2024-05-11T01:01:01.691669+0000","last_unstale":"2024-05-11T01:01:02.705339+0000","last_undegraded":"2024-05-11T01:01:02.705339+0000","last_fullsized":"2024-05-11T01:01:02.705339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:10:39.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677711+0000","last_change":"2024-05-11T01:01:01.691243+0000","last_active":"2024-05-11T01:01:02.677711+0000","last_peered":"2024-05-11T01:01:02.677711+0000","last_clean":"2024-05-11T01:01:02.677711+0000","last_became_active":"2024-05-11T01:01:01.690974+0000","last_became_peered":"2024-05-11T01:01:01.690974+0000","last_unstale":"2024-05-11T01:01:02.677711+0000","last_undegraded":"2024-05-11T01:01:02.677711+0000","last_fullsized":"2024-05-11T01:01:02.677711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:25.558002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913441+0000","last_change":"2024-05-11T01:01:00.681960+0000","last_active":"2024-05-11T01:01:02.913441+0000","last_peered":"2024-05-11T01:01:02.913441+0000","last_clean":"2024-05-11T01:01:02.913441+0000","last_became_active":"2024-05-11T01:01:00.681658+0000","last_became_peered":"2024-05-11T01:01:00.681658+0000","last_unstale":"2024-05-11T01:01:02.913441+0000","last_undegraded":"2024-05-11T01:01:02.913441+0000","last_fullsized":"2024-05-11T01:01:02.913441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676953+0000","last_change":"2024-05-11T01:01:01.697201+0000","last_active":"2024-05-11T01:01:02.676953+0000","last_peered":"2024-05-11T01:01:02.676953+0000","last_clean":"2024-05-11T01:01:02.676953+0000","last_became_active":"2024-05-11T01:01:01.696525+0000","last_became_peered":"2024-05-11T01:01:01.696525+0000","last_unstale":"2024-05-11T01:01:02.676953+0000","last_undegraded":"2024-05-11T01:01:02.676953+0000","last_fullsized":"2024-05-11T01:01:02.676953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:27:35.463648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705370+0000","last_change":"2024-05-11T01:01:00.684578+0000","last_active":"2024-05-11T01:01:02.705370+0000","last_peered":"2024-05-11T01:01:02.705370+0000","last_clean":"2024-05-11T01:01:02.705370+0000","last_became_active":"2024-05-11T01:01:00.684303+0000","last_became_peered":"2024-05-11T01:01:00.684303+0000","last_unstale":"2024-05-11T01:01:02.705370+0000","last_undegraded":"2024-05-11T01:01:02.705370+0000","last_fullsized":"2024-05-11T01:01:02.705370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913278+0000","last_change":"2024-05-11T01:01:00.678144+0000","last_active":"2024-05-11T01:01:02.913278+0000","last_peered":"2024-05-11T01:01:02.913278+0000","last_clean":"2024-05-11T01:01:02.913278+0000","last_became_active":"2024-05-11T01:01:00.677790+0000","last_became_peered":"2024-05-11T01:01:00.677790+0000","last_unstale":"2024-05-11T01:01:02.913278+0000","last_undegraded":"2024-05-11T01:01:02.913278+0000","last_fullsized":"2024-05-11T01:01:02.913278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:38:37.156429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913068+0000","last_change":"2024-05-11T01:01:01.689344+0000","last_active":"2024-05-11T01:01:02.913068+0000","last_peered":"2024-05-11T01:01:02.913068+0000","last_clean":"2024-05-11T01:01:02.913068+0000","last_became_active":"2024-05-11T01:01:01.687984+0000","last_became_peered":"2024-05-11T01:01:01.687984+0000","last_unstale":"2024-05-11T01:01:02.913068+0000","last_undegraded":"2024-05-11T01:01:02.913068+0000","last_fullsized":"2024-05-11T01:01:02.913068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:27:35.463648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678187+0000","last_change":"2024-05-11T01:01:00.678981+0000","last_active":"2024-05-11T01:01:02.678187+0000","last_peered":"2024-05-11T01:01:02.678187+0000","last_clean":"2024-05-11T01:01:02.678187+0000","last_became_active":"2024-05-11T01:01:00.678672+0000","last_became_peered":"2024-05-11T01:01:00.678672+0000","last_unstale":"2024-05-11T01:01:02.678187+0000","last_undegraded":"2024-05-11T01:01:02.678187+0000","last_fullsized":"2024-05-11T01:01:02.678187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.702964+0000","last_change":"2024-05-11T01:01:01.694506+0000","last_active":"2024-05-11T01:01:02.702964+0000","last_peered":"2024-05-11T01:01:02.702964+0000","last_clean":"2024-05-11T01:01:02.702964+0000","last_became_active":"2024-05-11T01:01:01.694226+0000","last_became_peered":"2024-05-11T01:01:01.694226+0000","last_unstale":"2024-05-11T01:01:02.702964+0000","last_undegraded":"2024-05-11T01:01:02.702964+0000","last_fullsized":"2024-05-11T01:01:02.702964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705643+0000","last_change":"2024-05-11T01:01:00.679681+0000","last_active":"2024-05-11T01:01:02.705643+0000","last_peered":"2024-05-11T01:01:02.705643+0000","last_clean":"2024-05-11T01:01:02.705643+0000","last_became_active":"2024-05-11T01:01:00.679125+0000","last_became_peered":"2024-05-11T01:01:00.679125+0000","last_unstale":"2024-05-11T01:01:02.705643+0000","last_undegraded":"2024-05-11T01:01:02.705643+0000","last_fullsized":"2024-05-11T01:01:02.705643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678282+0000","last_change":"2024-05-11T01:01:01.690424+0000","last_active":"2024-05-11T01:01:02.678282+0000","last_peered":"2024-05-11T01:01:02.678282+0000","last_clean":"2024-05-11T01:01:02.678282+0000","last_became_active":"2024-05-11T01:01:01.690131+0000","last_became_peered":"2024-05-11T01:01:01.690131+0000","last_unstale":"2024-05-11T01:01:02.678282+0000","last_undegraded":"2024-05-11T01:01:02.678282+0000","last_fullsized":"2024-05-11T01:01:02.678282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:35:54.225824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678389+0000","last_change":"2024-05-11T01:01:00.676347+0000","last_active":"2024-05-11T01:01:02.678389+0000","last_peered":"2024-05-11T01:01:02.678389+0000","last_clean":"2024-05-11T01:01:02.678389+0000","last_became_active":"2024-05-11T01:01:00.676058+0000","last_became_peered":"2024-05-11T01:01:00.676058+0000","last_unstale":"2024-05-11T01:01:02.678389+0000","last_undegraded":"2024-05-11T01:01:02.678389+0000","last_fullsized":"2024-05-11T01:01:02.678389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677252+0000","last_change":"2024-05-11T01:01:01.685176+0000","last_active":"2024-05-11T01:01:02.677252+0000","last_peered":"2024-05-11T01:01:02.677252+0000","last_clean":"2024-05-11T01:01:02.677252+0000","last_became_active":"2024-05-11T01:01:01.684804+0000","last_became_peered":"2024-05-11T01:01:01.684804+0000","last_unstale":"2024-05-11T01:01:02.677252+0000","last_undegraded":"2024-05-11T01:01:02.677252+0000","last_fullsized":"2024-05-11T01:01:02.677252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703190+0000","last_change":"2024-05-11T01:01:00.673845+0000","last_active":"2024-05-11T01:01:02.703190+0000","last_peered":"2024-05-11T01:01:02.703190+0000","last_clean":"2024-05-11T01:01:02.703190+0000","last_became_active":"2024-05-11T01:01:00.673430+0000","last_became_peered":"2024-05-11T01:01:00.673430+0000","last_unstale":"2024-05-11T01:01:02.703190+0000","last_undegraded":"2024-05-11T01:01:02.703190+0000","last_fullsized":"2024-05-11T01:01:02.703190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678463+0000","last_change":"2024-05-11T01:01:01.695552+0000","last_active":"2024-05-11T01:01:02.678463+0000","last_peered":"2024-05-11T01:01:02.678463+0000","last_clean":"2024-05-11T01:01:02.678463+0000","last_became_active":"2024-05-11T01:01:01.693674+0000","last_became_peered":"2024-05-11T01:01:01.693674+0000","last_unstale":"2024-05-11T01:01:02.678463+0000","last_undegraded":"2024-05-11T01:01:02.678463+0000","last_fullsized":"2024-05-11T01:01:02.678463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:17.700359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705505+0000","last_change":"2024-05-11T01:01:00.683575+0000","last_active":"2024-05-11T01:01:02.705505+0000","last_peered":"2024-05-11T01:01:02.705505+0000","last_clean":"2024-05-11T01:01:02.705505+0000","last_became_active":"2024-05-11T01:01:00.683259+0000","last_became_peered":"2024-05-11T01:01:00.683259+0000","last_unstale":"2024-05-11T01:01:02.705505+0000","last_undegraded":"2024-05-11T01:01:02.705505+0000","last_fullsized":"2024-05-11T01:01:02.705505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793910+0000","last_change":"2024-05-11T01:01:01.699906+0000","last_active":"2024-05-11T01:01:02.793910+0000","last_peered":"2024-05-11T01:01:02.793910+0000","last_clean":"2024-05-11T01:01:02.793910+0000","last_became_active":"2024-05-11T01:01:01.699637+0000","last_became_peered":"2024-05-11T01:01:01.699637+0000","last_unstale":"2024-05-11T01:01:02.793910+0000","last_undegraded":"2024-05-11T01:01:02.793910+0000","last_fullsized":"2024-05-11T01:01:02.793910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:13:49.063574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677033+0000","last_change":"2024-05-11T01:01:00.684668+0000","last_active":"2024-05-11T01:01:02.677033+0000","last_peered":"2024-05-11T01:01:02.677033+0000","last_clean":"2024-05-11T01:01:02.677033+0000","last_became_active":"2024-05-11T01:01:00.684364+0000","last_became_peered":"2024-05-11T01:01:00.684364+0000","last_unstale":"2024-05-11T01:01:02.677033+0000","last_undegraded":"2024-05-11T01:01:02.677033+0000","last_fullsized":"2024-05-11T01:01:02.677033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:54.494836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676757+0000","last_change":"2024-05-11T01:01:01.686608+0000","last_active":"2024-05-11T01:01:02.676757+0000","last_peered":"2024-05-11T01:01:02.676757+0000","last_clean":"2024-05-11T01:01:02.676757+0000","last_became_active":"2024-05-11T01:01:01.686333+0000","last_became_peered":"2024-05-11T01:01:01.686333+0000","last_unstale":"2024-05-11T01:01:02.676757+0000","last_undegraded":"2024-05-11T01:01:02.676757+0000","last_fullsized":"2024-05-11T01:01:02.676757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:51.874581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678282+0000","last_change":"2024-05-11T01:01:00.684592+0000","last_active":"2024-05-11T01:01:02.678282+0000","last_peered":"2024-05-11T01:01:02.678282+0000","last_clean":"2024-05-11T01:01:02.678282+0000","last_became_active":"2024-05-11T01:01:00.683709+0000","last_became_peered":"2024-05-11T01:01:00.683709+0000","last_unstale":"2024-05-11T01:01:02.678282+0000","last_undegraded":"2024-05-11T01:01:02.678282+0000","last_fullsized":"2024-05-11T01:01:02.678282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793773+0000","last_change":"2024-05-11T01:01:01.699198+0000","last_active":"2024-05-11T01:01:02.793773+0000","last_peered":"2024-05-11T01:01:02.793773+0000","last_clean":"2024-05-11T01:01:02.793773+0000","last_became_active":"2024-05-11T01:01:01.698959+0000","last_became_peered":"2024-05-11T01:01:01.698959+0000","last_unstale":"2024-05-11T01:01:02.793773+0000","last_undegraded":"2024-05-11T01:01:02.793773+0000","last_fullsized":"2024-05-11T01:01:02.793773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:32.857796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705344+0000","last_change":"2024-05-11T01:01:00.684714+0000","last_active":"2024-05-11T01:01:02.705344+0000","last_peered":"2024-05-11T01:01:02.705344+0000","last_clean":"2024-05-11T01:01:02.705344+0000","last_became_active":"2024-05-11T01:01:00.684454+0000","last_became_peered":"2024-05-11T01:01:00.684454+0000","last_unstale":"2024-05-11T01:01:02.705344+0000","last_undegraded":"2024-05-11T01:01:02.705344+0000","last_fullsized":"2024-05-11T01:01:02.705344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705449+0000","last_change":"2024-05-11T01:01:01.691666+0000","last_active":"2024-05-11T01:01:02.705449+0000","last_peered":"2024-05-11T01:01:02.705449+0000","last_clean":"2024-05-11T01:01:02.705449+0000","last_became_active":"2024-05-11T01:01:01.691368+0000","last_became_peered":"2024-05-11T01:01:01.691368+0000","last_unstale":"2024-05-11T01:01:02.705449+0000","last_undegraded":"2024-05-11T01:01:02.705449+0000","last_fullsized":"2024-05-11T01:01:02.705449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912707+0000","last_change":"2024-05-11T01:01:00.680506+0000","last_active":"2024-05-11T01:01:02.912707+0000","last_peered":"2024-05-11T01:01:02.912707+0000","last_clean":"2024-05-11T01:01:02.912707+0000","last_became_active":"2024-05-11T01:01:00.680198+0000","last_became_peered":"2024-05-11T01:01:00.680198+0000","last_unstale":"2024-05-11T01:01:02.912707+0000","last_undegraded":"2024-05-11T01:01:02.912707+0000","last_fullsized":"2024-05-11T01:01:02.912707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793544+0000","last_change":"2024-05-11T01:01:01.698747+0000","last_active":"2024-05-11T01:01:02.793544+0000","last_peered":"2024-05-11T01:01:02.793544+0000","last_clean":"2024-05-11T01:01:02.793544+0000","last_became_active":"2024-05-11T01:01:01.698482+0000","last_became_peered":"2024-05-11T01:01:01.698482+0000","last_unstale":"2024-05-11T01:01:02.793544+0000","last_undegraded":"2024-05-11T01:01:02.793544+0000","last_fullsized":"2024-05-11T01:01:02.793544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:35:54.225824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678878+0000","last_change":"2024-05-11T01:01:00.686573+0000","last_active":"2024-05-11T01:01:02.678878+0000","last_peered":"2024-05-11T01:01:02.678878+0000","last_clean":"2024-05-11T01:01:02.678878+0000","last_became_active":"2024-05-11T01:01:00.686222+0000","last_became_peered":"2024-05-11T01:01:00.686222+0000","last_unstale":"2024-05-11T01:01:02.678878+0000","last_undegraded":"2024-05-11T01:01:02.678878+0000","last_fullsized":"2024-05-11T01:01:02.678878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:20:07.656810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794105+0000","last_change":"2024-05-11T01:01:01.697983+0000","last_active":"2024-05-11T01:01:02.794105+0000","last_peered":"2024-05-11T01:01:02.794105+0000","last_clean":"2024-05-11T01:01:02.794105+0000","last_became_active":"2024-05-11T01:01:01.697645+0000","last_became_peered":"2024-05-11T01:01:01.697645+0000","last_unstale":"2024-05-11T01:01:02.794105+0000","last_undegraded":"2024-05-11T01:01:02.794105+0000","last_fullsized":"2024-05-11T01:01:02.794105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:12.996809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913020+0000","last_change":"2024-05-11T01:01:00.684168+0000","last_active":"2024-05-11T01:01:02.913020+0000","last_peered":"2024-05-11T01:01:02.913020+0000","last_clean":"2024-05-11T01:01:02.913020+0000","last_became_active":"2024-05-11T01:01:00.683839+0000","last_became_peered":"2024-05-11T01:01:00.683839+0000","last_unstale":"2024-05-11T01:01:02.913020+0000","last_undegraded":"2024-05-11T01:01:02.913020+0000","last_fullsized":"2024-05-11T01:01:02.913020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794232+0000","last_change":"2024-05-11T01:01:01.698302+0000","last_active":"2024-05-11T01:01:02.794232+0000","last_peered":"2024-05-11T01:01:02.794232+0000","last_clean":"2024-05-11T01:01:02.794232+0000","last_became_active":"2024-05-11T01:01:01.697989+0000","last_became_peered":"2024-05-11T01:01:01.697989+0000","last_unstale":"2024-05-11T01:01:02.794232+0000","last_undegraded":"2024-05-11T01:01:02.794232+0000","last_fullsized":"2024-05-11T01:01:02.794232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:20:37.893643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703231+0000","last_change":"2024-05-11T01:01:00.681331+0000","last_active":"2024-05-11T01:01:02.703231+0000","last_peered":"2024-05-11T01:01:02.703231+0000","last_clean":"2024-05-11T01:01:02.703231+0000","last_became_active":"2024-05-11T01:01:00.680679+0000","last_became_peered":"2024-05-11T01:01:00.680679+0000","last_unstale":"2024-05-11T01:01:02.703231+0000","last_undegraded":"2024-05-11T01:01:02.703231+0000","last_fullsized":"2024-05-11T01:01:02.703231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678940+0000","last_change":"2024-05-11T01:01:01.687504+0000","last_active":"2024-05-11T01:01:02.678940+0000","last_peered":"2024-05-11T01:01:02.678940+0000","last_clean":"2024-05-11T01:01:02.678940+0000","last_became_active":"2024-05-11T01:01:01.686851+0000","last_became_peered":"2024-05-11T01:01:01.686851+0000","last_unstale":"2024-05-11T01:01:02.678940+0000","last_undegraded":"2024-05-11T01:01:02.678940+0000","last_fullsized":"2024-05-11T01:01:02.678940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:00:03.997273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913094+0000","last_change":"2024-05-11T01:01:00.682797+0000","last_active":"2024-05-11T01:01:02.913094+0000","last_peered":"2024-05-11T01:01:02.913094+0000","last_clean":"2024-05-11T01:01:02.913094+0000","last_became_active":"2024-05-11T01:01:00.682282+0000","last_became_peered":"2024-05-11T01:01:00.682282+0000","last_unstale":"2024-05-11T01:01:02.913094+0000","last_undegraded":"2024-05-11T01:01:02.913094+0000","last_fullsized":"2024-05-11T01:01:02.913094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:00:17.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793948+0000","last_change":"2024-05-11T01:01:01.705606+0000","last_active":"2024-05-11T01:01:02.793948+0000","last_peered":"2024-05-11T01:01:02.793948+0000","last_clean":"2024-05-11T01:01:02.793948+0000","last_became_active":"2024-05-11T01:01:01.705324+0000","last_became_peered":"2024-05-11T01:01:01.705324+0000","last_unstale":"2024-05-11T01:01:02.793948+0000","last_undegraded":"2024-05-11T01:01:02.793948+0000","last_fullsized":"2024-05-11T01:01:02.793948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:31.889721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793424+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:01:02.793424+0000","last_peered":"2024-05-11T01:01:02.793424+0000","last_clean":"2024-05-11T01:01:02.793424+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:01:02.793424+0000","last_undegraded":"2024-05-11T01:01:02.793424+0000","last_fullsized":"2024-05-11T01:01:02.793424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677503+0000","last_change":"2024-05-11T01:01:00.679561+0000","last_active":"2024-05-11T01:01:02.677503+0000","last_peered":"2024-05-11T01:01:02.677503+0000","last_clean":"2024-05-11T01:01:02.677503+0000","last_became_active":"2024-05-11T01:01:00.679241+0000","last_became_peered":"2024-05-11T01:01:00.679241+0000","last_unstale":"2024-05-11T01:01:02.677503+0000","last_undegraded":"2024-05-11T01:01:02.677503+0000","last_fullsized":"2024-05-11T01:01:02.677503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913407+0000","last_change":"2024-05-11T01:01:01.693541+0000","last_active":"2024-05-11T01:01:02.913407+0000","last_peered":"2024-05-11T01:01:02.913407+0000","last_clean":"2024-05-11T01:01:02.913407+0000","last_became_active":"2024-05-11T01:01:01.693033+0000","last_became_peered":"2024-05-11T01:01:01.693033+0000","last_unstale":"2024-05-11T01:01:02.913407+0000","last_undegraded":"2024-05-11T01:01:02.913407+0000","last_fullsized":"2024-05-11T01:01:02.913407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793351+0000","last_change":"2024-05-11T01:01:00.689701+0000","last_active":"2024-05-11T01:01:02.793351+0000","last_peered":"2024-05-11T01:01:02.793351+0000","last_clean":"2024-05-11T01:01:02.793351+0000","last_became_active":"2024-05-11T01:01:00.689439+0000","last_became_peered":"2024-05-11T01:01:00.689439+0000","last_unstale":"2024-05-11T01:01:02.793351+0000","last_undegraded":"2024-05-11T01:01:02.793351+0000","last_fullsized":"2024-05-11T01:01:02.793351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:10:33.194315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676839+0000","last_change":"2024-05-11T01:01:01.696524+0000","last_active":"2024-05-11T01:01:02.676839+0000","last_peered":"2024-05-11T01:01:02.676839+0000","last_clean":"2024-05-11T01:01:02.676839+0000","last_became_active":"2024-05-11T01:01:01.696245+0000","last_became_peered":"2024-05-11T01:01:01.696245+0000","last_unstale":"2024-05-11T01:01:02.676839+0000","last_undegraded":"2024-05-11T01:01:02.676839+0000","last_fullsized":"2024-05-11T01:01:02.676839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:33.546403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912909+0000","last_change":"2024-05-11T01:01:00.683932+0000","last_active":"2024-05-11T01:01:02.912909+0000","last_peered":"2024-05-11T01:01:02.912909+0000","last_clean":"2024-05-11T01:01:02.912909+0000","last_became_active":"2024-05-11T01:01:00.683601+0000","last_became_peered":"2024-05-11T01:01:00.683601+0000","last_unstale":"2024-05-11T01:01:02.912909+0000","last_undegraded":"2024-05-11T01:01:02.912909+0000","last_fullsized":"2024-05-11T01:01:02.912909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.702979+0000","last_change":"2024-05-11T01:01:01.691231+0000","last_active":"2024-05-11T01:01:02.702979+0000","last_peered":"2024-05-11T01:01:02.702979+0000","last_clean":"2024-05-11T01:01:02.702979+0000","last_became_active":"2024-05-11T01:01:01.690969+0000","last_became_peered":"2024-05-11T01:01:01.690969+0000","last_unstale":"2024-05-11T01:01:02.702979+0000","last_undegraded":"2024-05-11T01:01:02.702979+0000","last_fullsized":"2024-05-11T01:01:02.702979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678402+0000","last_change":"2024-05-11T01:01:00.682079+0000","last_active":"2024-05-11T01:01:02.678402+0000","last_peered":"2024-05-11T01:01:02.678402+0000","last_clean":"2024-05-11T01:01:02.678402+0000","last_became_active":"2024-05-11T01:01:00.681773+0000","last_became_peered":"2024-05-11T01:01:00.681773+0000","last_unstale":"2024-05-11T01:01:02.678402+0000","last_undegraded":"2024-05-11T01:01:02.678402+0000","last_fullsized":"2024-05-11T01:01:02.678402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.679107+0000","last_change":"2024-05-11T01:01:01.695539+0000","last_active":"2024-05-11T01:01:02.679107+0000","last_peered":"2024-05-11T01:01:02.679107+0000","last_clean":"2024-05-11T01:01:02.679107+0000","last_became_active":"2024-05-11T01:01:01.695235+0000","last_became_peered":"2024-05-11T01:01:01.695235+0000","last_unstale":"2024-05-11T01:01:02.679107+0000","last_undegraded":"2024-05-11T01:01:02.679107+0000","last_fullsized":"2024-05-11T01:01:02.679107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:44.383036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703144+0000","last_change":"2024-05-11T01:01:00.680453+0000","last_active":"2024-05-11T01:01:02.703144+0000","last_peered":"2024-05-11T01:01:02.703144+0000","last_clean":"2024-05-11T01:01:02.703144+0000","last_became_active":"2024-05-11T01:01:00.679559+0000","last_became_peered":"2024-05-11T01:01:00.679559+0000","last_unstale":"2024-05-11T01:01:02.703144+0000","last_undegraded":"2024-05-11T01:01:02.703144+0000","last_fullsized":"2024-05-11T01:01:02.703144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:10:33.194315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678735+0000","last_change":"2024-05-11T01:01:01.687198+0000","last_active":"2024-05-11T01:01:02.678735+0000","last_peered":"2024-05-11T01:01:02.678735+0000","last_clean":"2024-05-11T01:01:02.678735+0000","last_became_active":"2024-05-11T01:01:01.686860+0000","last_became_peered":"2024-05-11T01:01:01.686860+0000","last_unstale":"2024-05-11T01:01:02.678735+0000","last_undegraded":"2024-05-11T01:01:02.678735+0000","last_fullsized":"2024-05-11T01:01:02.678735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:31.889721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794088+0000","last_change":"2024-05-11T01:01:00.688488+0000","last_active":"2024-05-11T01:01:02.794088+0000","last_peered":"2024-05-11T01:01:02.794088+0000","last_clean":"2024-05-11T01:01:02.794088+0000","last_became_active":"2024-05-11T01:01:00.688109+0000","last_became_peered":"2024-05-11T01:01:00.688109+0000","last_unstale":"2024-05-11T01:01:02.794088+0000","last_undegraded":"2024-05-11T01:01:02.794088+0000","last_fullsized":"2024-05-11T01:01:02.794088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:32.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703144+0000","last_change":"2024-05-11T01:01:01.689340+0000","last_active":"2024-05-11T01:01:02.703144+0000","last_peered":"2024-05-11T01:01:02.703144+0000","last_clean":"2024-05-11T01:01:02.703144+0000","last_became_active":"2024-05-11T01:01:01.689052+0000","last_became_peered":"2024-05-11T01:01:01.689052+0000","last_unstale":"2024-05-11T01:01:02.703144+0000","last_undegraded":"2024-05-11T01:01:02.703144+0000","last_fullsized":"2024-05-11T01:01:02.703144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703207+0000","last_change":"2024-05-11T01:01:00.678958+0000","last_active":"2024-05-11T01:01:02.703207+0000","last_peered":"2024-05-11T01:01:02.703207+0000","last_clean":"2024-05-11T01:01:02.703207+0000","last_became_active":"2024-05-11T01:01:00.678608+0000","last_became_peered":"2024-05-11T01:01:00.678608+0000","last_unstale":"2024-05-11T01:01:02.703207+0000","last_undegraded":"2024-05-11T01:01:02.703207+0000","last_fullsized":"2024-05-11T01:01:02.703207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:58.764462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913742+0000","last_change":"2024-05-11T01:01:01.686046+0000","last_active":"2024-05-11T01:01:02.913742+0000","last_peered":"2024-05-11T01:01:02.913742+0000","last_clean":"2024-05-11T01:01:02.913742+0000","last_became_active":"2024-05-11T01:01:01.685771+0000","last_became_peered":"2024-05-11T01:01:01.685771+0000","last_unstale":"2024-05-11T01:01:02.913742+0000","last_undegraded":"2024-05-11T01:01:02.913742+0000","last_fullsized":"2024-05-11T01:01:02.913742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:36:33.062657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705667+0000","last_change":"2024-05-11T01:01:00.686048+0000","last_active":"2024-05-11T01:01:02.705667+0000","last_peered":"2024-05-11T01:01:02.705667+0000","last_clean":"2024-05-11T01:01:02.705667+0000","last_became_active":"2024-05-11T01:01:00.685215+0000","last_became_peered":"2024-05-11T01:01:00.685215+0000","last_unstale":"2024-05-11T01:01:02.705667+0000","last_undegraded":"2024-05-11T01:01:02.705667+0000","last_fullsized":"2024-05-11T01:01:02.705667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677114+0000","last_change":"2024-05-11T01:01:01.695783+0000","last_active":"2024-05-11T01:01:02.677114+0000","last_peered":"2024-05-11T01:01:02.677114+0000","last_clean":"2024-05-11T01:01:02.677114+0000","last_became_active":"2024-05-11T01:01:01.695264+0000","last_became_peered":"2024-05-11T01:01:01.695264+0000","last_unstale":"2024-05-11T01:01:02.677114+0000","last_undegraded":"2024-05-11T01:01:02.677114+0000","last_fullsized":"2024-05-11T01:01:02.677114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793321+0000","last_change":"2024-05-11T01:01:00.698030+0000","last_active":"2024-05-11T01:01:02.793321+0000","last_peered":"2024-05-11T01:01:02.793321+0000","last_clean":"2024-05-11T01:01:02.793321+0000","last_became_active":"2024-05-11T01:01:00.697776+0000","last_became_peered":"2024-05-11T01:01:00.697776+0000","last_unstale":"2024-05-11T01:01:02.793321+0000","last_undegraded":"2024-05-11T01:01:02.793321+0000","last_fullsized":"2024-05-11T01:01:02.793321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677953+0000","last_change":"2024-05-11T01:01:01.696098+0000","last_active":"2024-05-11T01:01:02.677953+0000","last_peered":"2024-05-11T01:01:02.677953+0000","last_clean":"2024-05-11T01:01:02.677953+0000","last_became_active":"2024-05-11T01:01:01.695670+0000","last_became_peered":"2024-05-11T01:01:01.695670+0000","last_unstale":"2024-05-11T01:01:02.677953+0000","last_undegraded":"2024-05-11T01:01:02.677953+0000","last_fullsized":"2024-05-11T01:01:02.677953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:49.477561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703023+0000","last_change":"2024-05-11T01:01:00.680814+0000","last_active":"2024-05-11T01:01:02.703023+0000","last_peered":"2024-05-11T01:01:02.703023+0000","last_clean":"2024-05-11T01:01:02.703023+0000","last_became_active":"2024-05-11T01:01:00.680415+0000","last_became_peered":"2024-05-11T01:01:00.680415+0000","last_unstale":"2024-05-11T01:01:02.703023+0000","last_undegraded":"2024-05-11T01:01:02.703023+0000","last_fullsized":"2024-05-11T01:01:02.703023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913100+0000","last_change":"2024-05-11T01:01:01.690240+0000","last_active":"2024-05-11T01:01:02.913100+0000","last_peered":"2024-05-11T01:01:02.913100+0000","last_clean":"2024-05-11T01:01:02.913100+0000","last_became_active":"2024-05-11T01:01:01.689938+0000","last_became_peered":"2024-05-11T01:01:01.689938+0000","last_unstale":"2024-05-11T01:01:02.913100+0000","last_undegraded":"2024-05-11T01:01:02.913100+0000","last_fullsized":"2024-05-11T01:01:02.913100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:10:39.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703036+0000","last_change":"2024-05-11T01:01:00.682185+0000","last_active":"2024-05-11T01:01:02.703036+0000","last_peered":"2024-05-11T01:01:02.703036+0000","last_clean":"2024-05-11T01:01:02.703036+0000","last_became_active":"2024-05-11T01:01:00.681773+0000","last_became_peered":"2024-05-11T01:01:00.681773+0000","last_unstale":"2024-05-11T01:01:02.703036+0000","last_undegraded":"2024-05-11T01:01:02.703036+0000","last_fullsized":"2024-05-11T01:01:02.703036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913467+0000","last_change":"2024-05-11T01:01:01.688107+0000","last_active":"2024-05-11T01:01:02.913467+0000","last_peered":"2024-05-11T01:01:02.913467+0000","last_clean":"2024-05-11T01:01:02.913467+0000","last_became_active":"2024-05-11T01:01:01.686201+0000","last_became_peered":"2024-05-11T01:01:01.686201+0000","last_unstale":"2024-05-11T01:01:02.913467+0000","last_undegraded":"2024-05-11T01:01:02.913467+0000","last_fullsized":"2024-05-11T01:01:02.913467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793707+0000","last_change":"2024-05-11T01:01:00.683637+0000","last_active":"2024-05-11T01:01:02.793707+0000","last_peered":"2024-05-11T01:01:02.793707+0000","last_clean":"2024-05-11T01:01:02.793707+0000","last_became_active":"2024-05-11T01:01:00.683373+0000","last_became_peered":"2024-05-11T01:01:00.683373+0000","last_unstale":"2024-05-11T01:01:02.793707+0000","last_undegraded":"2024-05-11T01:01:02.793707+0000","last_fullsized":"2024-05-11T01:01:02.793707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:58.764462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705465+0000","last_change":"2024-05-11T01:01:01.695407+0000","last_active":"2024-05-11T01:01:02.705465+0000","last_peered":"2024-05-11T01:01:02.705465+0000","last_clean":"2024-05-11T01:01:02.705465+0000","last_became_active":"2024-05-11T01:01:01.695012+0000","last_became_peered":"2024-05-11T01:01:01.695012+0000","last_unstale":"2024-05-11T01:01:02.705465+0000","last_undegraded":"2024-05-11T01:01:02.705465+0000","last_fullsized":"2024-05-11T01:01:02.705465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678475+0000","last_change":"2024-05-11T01:01:01.698832+0000","last_active":"2024-05-11T01:01:02.678475+0000","last_peered":"2024-05-11T01:01:02.678475+0000","last_clean":"2024-05-11T01:01:02.678475+0000","last_became_active":"2024-05-11T01:01:01.698487+0000","last_became_peered":"2024-05-11T01:01:01.698487+0000","last_unstale":"2024-05-11T01:01:02.678475+0000","last_undegraded":"2024-05-11T01:01:02.678475+0000","last_fullsized":"2024-05-11T01:01:02.678475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793796+0000","last_change":"2024-05-11T01:01:00.690097+0000","last_active":"2024-05-11T01:01:02.793796+0000","last_peered":"2024-05-11T01:01:02.793796+0000","last_clean":"2024-05-11T01:01:02.793796+0000","last_became_active":"2024-05-11T01:01:00.689772+0000","last_became_peered":"2024-05-11T01:01:00.689772+0000","last_unstale":"2024-05-11T01:01:02.793796+0000","last_undegraded":"2024-05-11T01:01:02.793796+0000","last_fullsized":"2024-05-11T01:01:02.793796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678776+0000","last_change":"2024-05-11T01:01:01.699646+0000","last_active":"2024-05-11T01:01:02.678776+0000","last_peered":"2024-05-11T01:01:02.678776+0000","last_clean":"2024-05-11T01:01:02.678776+0000","last_became_active":"2024-05-11T01:01:01.699388+0000","last_became_peered":"2024-05-11T01:01:01.699388+0000","last_unstale":"2024-05-11T01:01:02.678776+0000","last_undegraded":"2024-05-11T01:01:02.678776+0000","last_fullsized":"2024-05-11T01:01:02.678776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:12.996809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793982+0000","last_change":"2024-05-11T01:01:00.688793+0000","last_active":"2024-05-11T01:01:02.793982+0000","last_peered":"2024-05-11T01:01:02.793982+0000","last_clean":"2024-05-11T01:01:02.793982+0000","last_became_active":"2024-05-11T01:01:00.688279+0000","last_became_peered":"2024-05-11T01:01:00.688279+0000","last_unstale":"2024-05-11T01:01:02.793982+0000","last_undegraded":"2024-05-11T01:01:02.793982+0000","last_fullsized":"2024-05-11T01:01:02.793982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:50.870500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677179+0000","last_change":"2024-05-11T01:01:01.682113+0000","last_active":"2024-05-11T01:01:02.677179+0000","last_peered":"2024-05-11T01:01:02.677179+0000","last_clean":"2024-05-11T01:01:02.677179+0000","last_became_active":"2024-05-11T01:01:01.681706+0000","last_became_peered":"2024-05-11T01:01:01.681706+0000","last_unstale":"2024-05-11T01:01:02.677179+0000","last_undegraded":"2024-05-11T01:01:02.677179+0000","last_fullsized":"2024-05-11T01:01:02.677179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913646+0000","last_change":"2024-05-11T01:01:00.680118+0000","last_active":"2024-05-11T01:01:02.913646+0000","last_peered":"2024-05-11T01:01:02.913646+0000","last_clean":"2024-05-11T01:01:02.913646+0000","last_became_active":"2024-05-11T01:01:00.679809+0000","last_became_peered":"2024-05-11T01:01:00.679809+0000","last_unstale":"2024-05-11T01:01:02.913646+0000","last_undegraded":"2024-05-11T01:01:02.913646+0000","last_fullsized":"2024-05-11T01:01:02.913646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:12:31.764619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913693+0000","last_change":"2024-05-11T01:01:01.692697+0000","last_active":"2024-05-11T01:01:02.913693+0000","last_peered":"2024-05-11T01:01:02.913693+0000","last_clean":"2024-05-11T01:01:02.913693+0000","last_became_active":"2024-05-11T01:01:01.691026+0000","last_became_peered":"2024-05-11T01:01:01.691026+0000","last_unstale":"2024-05-11T01:01:02.913693+0000","last_undegraded":"2024-05-11T01:01:02.913693+0000","last_fullsized":"2024-05-11T01:01:02.913693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677208+0000","last_change":"2024-05-11T01:01:00.690191+0000","last_active":"2024-05-11T01:01:02.677208+0000","last_peered":"2024-05-11T01:01:02.677208+0000","last_clean":"2024-05-11T01:01:02.677208+0000","last_became_active":"2024-05-11T01:01:00.689804+0000","last_became_peered":"2024-05-11T01:01:00.689804+0000","last_unstale":"2024-05-11T01:01:02.677208+0000","last_undegraded":"2024-05-11T01:01:02.677208+0000","last_fullsized":"2024-05-11T01:01:02.677208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:13.659371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:01:10.554 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T01:01:10.554 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-11T01:01:10.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-11T01:01:10.554 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-11T01:01:10.555 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-11T01:01:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.692+0000 7fb237883640 1 Processor -- start 2024-05-11T01:01:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.692+0000 7fb237883640 1 -- start start 2024-05-11T01:01:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb237883640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb23015f430 0x7fb23015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb237883640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb2301602b0 0x7fb2300d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb237883640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb230163250 con 0x7fb230160720 2024-05-11T01:01:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb237883640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb230163530 con 0x7fb2301602b0 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb237883640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb230163810 con 0x7fb23015f430 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb234e1e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb2301602b0 0x7fb2300d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb234e1e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb2301602b0 0x7fb2300d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60428/0 (socket says 172.21.15.52:60428) 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb234e1e640 1 -- 172.21.15.52:0/3499070531 learned_addr learned my addr 172.21.15.52:0/3499070531 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb235e20640 1 --1- 172.21.15.52:0/3499070531 >> v1:172.21.15.52:6789/0 conn(0x7fb230160720 0x7fb2300d2090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44666/0 (socket says 172.21.15.52:44666) 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb23561f640 1 --2- 172.21.15.52:0/3499070531 >> v2:172.21.15.119:3300/0 conn(0x7fb23015f430 0x7fb23015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb234e1e640 1 -- 172.21.15.52:0/3499070531 >> v2:172.21.15.119:3300/0 conn(0x7fb23015f430 msgr2=0x7fb23015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.693+0000 7fb234e1e640 1 --2- 172.21.15.52:0/3499070531 >> v2:172.21.15.119:3300/0 conn(0x7fb23015f430 0x7fb23015fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.694+0000 7fb234e1e640 1 -- 172.21.15.52:0/3499070531 >> v1:172.21.15.52:6789/0 conn(0x7fb230160720 legacy=0x7fb2300d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.694+0000 7fb234e1e640 1 -- 172.21.15.52:0/3499070531 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb230163af0 con 0x7fb2301602b0 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.694+0000 7fb223fff640 1 -- 172.21.15.52:0/3499070531 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398499677 0 0) 0x7fb230163250 con 0x7fb230160720 2024-05-11T01:01:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.694+0000 7fb234e1e640 1 --2- 172.21.15.52:0/3499070531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 0x7fb2300d1b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b6b80 tx=0x7fb22c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9d220363d60aad server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.694+0000 7fb223fff640 1 -- 172.21.15.52:0/3499070531 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c037070 con 0x7fb2301602b0 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.694+0000 7fb223fff640 1 -- 172.21.15.52:0/3499070531 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c032440 con 0x7fb2301602b0 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.694+0000 7fb223fff640 1 -- 172.21.15.52:0/3499070531 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c032810 con 0x7fb2301602b0 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.695+0000 7fb237883640 1 -- 172.21.15.52:0/3499070531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 msgr2=0x7fb2300d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.695+0000 7fb237883640 1 --2- 172.21.15.52:0/3499070531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 0x7fb2300d1b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b6b80 tx=0x7fb22c02a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.695+0000 7fb237883640 1 -- 172.21.15.52:0/3499070531 shutdown_connections 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.695+0000 7fb237883640 1 --2- 172.21.15.52:0/3499070531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 0x7fb2300d1b50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.695+0000 7fb237883640 1 --2- 172.21.15.52:0/3499070531 >> v2:172.21.15.119:3300/0 conn(0x7fb23015f430 0x7fb23015fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.695+0000 7fb237883640 1 -- 172.21.15.52:0/3499070531 >> 172.21.15.52:0/3499070531 conn(0x7fb23008eb70 msgr2=0x7fb23009e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.695+0000 7fb237883640 1 -- 172.21.15.52:0/3499070531 shutdown_connections 2024-05-11T01:01:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.695+0000 7fb237883640 1 -- 172.21.15.52:0/3499070531 wait complete. 2024-05-11T01:01:10.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.696+0000 7fb237883640 1 Processor -- start 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.696+0000 7fb237883640 1 -- start start 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb237883640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb23015f430 0x7fb2301525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb237883640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 0x7fb230161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb237883640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb230160720 0x7fb230162340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb237883640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb230163030 con 0x7fb2301602b0 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb237883640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb230162eb0 con 0x7fb230160720 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb237883640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb2300d27a0 con 0x7fb23015f430 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb234e1e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 0x7fb230161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb234e1e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 0x7fb230161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60434/0 (socket says 172.21.15.52:60434) 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb234e1e640 1 -- 172.21.15.52:0/1439165689 learned_addr learned my addr 172.21.15.52:0/1439165689 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb23561f640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb23015f430 0x7fb2301525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb235e20640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb230160720 0x7fb230162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb234e1e640 1 -- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb23015f430 msgr2=0x7fb2301525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb234e1e640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb23015f430 0x7fb2301525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb234e1e640 1 -- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb230160720 msgr2=0x7fb230162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb234e1e640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb230160720 0x7fb230162340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb234e1e640 1 -- 172.21.15.52:0/1439165689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2300cf010 con 0x7fb2301602b0 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.697+0000 7fb234e1e640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 0x7fb230161e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb22c02a220 tx=0x7fb22c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.698+0000 7fb221ffb640 1 -- 172.21.15.52:0/1439165689 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c041020 con 0x7fb2301602b0 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.698+0000 7fb221ffb640 1 -- 172.21.15.52:0/1439165689 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c0306c0 con 0x7fb2301602b0 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.698+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2300cdf70 con 0x7fb2301602b0 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.698+0000 7fb221ffb640 1 -- 172.21.15.52:0/1439165689 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c037040 con 0x7fb2301602b0 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.698+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2300ce530 con 0x7fb2301602b0 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.699+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb230069780 con 0x7fb2301602b0 2024-05-11T01:01:10.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.700+0000 7fb221ffb640 1 -- 172.21.15.52:0/1439165689 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb22c044020 con 0x7fb2301602b0 2024-05-11T01:01:10.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.700+0000 7fb221ffb640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2140762d0 0x7fb214078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:10.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.700+0000 7fb221ffb640 1 -- 172.21.15.52:0/1439165689 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb22c0b7d70 con 0x7fb2301602b0 2024-05-11T01:01:10.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.700+0000 7fb23561f640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2140762d0 0x7fb214078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:10.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.701+0000 7fb23561f640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2140762d0 0x7fb214078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb224006fd0 tx=0x7fb224008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:10.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.704+0000 7fb221ffb640 1 -- 172.21.15.52:0/1439165689 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb22c04b030 con 0x7fb2301602b0 2024-05-11T01:01:10.867 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:01:10.869 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:01:10.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.945+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb2300b4ba0 con 0x7fb2301602b0 2024-05-11T01:01:10.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.948+0000 7fb221ffb640 1 -- 172.21.15.52:0/1439165689 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+189528 (secure 0 0 0) 0x7fb22c081760 con 0x7fb2301602b0 2024-05-11T01:01:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:10.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2140762d0 msgr2=0x7fb214078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2140762d0 0x7fb214078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb224006fd0 tx=0x7fb224008040 comp rx=0 tx=0).stop 2024-05-11T01:01:10.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 msgr2=0x7fb230161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:10.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 0x7fb230161e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb22c02a220 tx=0x7fb22c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 shutdown_connections 2024-05-11T01:01:10.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2140762d0 0x7fb214078790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb230160720 0x7fb230162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2301602b0 0x7fb230161e00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 --2- 172.21.15.52:0/1439165689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb23015f430 0x7fb2301525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:10.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 >> 172.21.15.52:0/1439165689 conn(0x7fb23008eb70 msgr2=0x7fb23006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:10.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 shutdown_connections 2024-05-11T01:01:10.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:10.951+0000 7fb237883640 1 -- 172.21.15.52:0/1439165689 wait complete. 2024-05-11T01:01:10.962 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3083030853},{"type":"v1","addr":"172.21.15.119:6833","nonce":3083030853}]},"active_addr":"172.21.15.119:6833/3083030853","active_change":"2024-05-11T01:00:49.501333+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1584246087}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1530920757}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1820176553}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1042890889}]}]} 2024-05-11T01:01:10.967 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-11T01:01:10.967 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-11T01:01:10.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:11.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.105+0000 7f7be59dd640 1 Processor -- start 2024-05-11T01:01:11.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.105+0000 7f7be59dd640 1 -- start start 2024-05-11T01:01:11.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7be59dd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7be01576a0 0x7f7be00add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7be59dd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 0x7f7be00af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7be59dd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7be015e1d0 0x7f7be00afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7be59dd640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7be00b03d0 con 0x7f7be015e1d0 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7be59dd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7be0160c50 con 0x7f7be01576a0 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7be59dd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7be0160f30 con 0x7f7be00ae290 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bdf7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7be015e1d0 0x7f7be00afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bdf7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7be015e1d0 0x7f7be00afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60440/0 (socket says 172.21.15.52:60440) 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bdf7fe640 1 -- 172.21.15.52:0/4162713686 learned_addr learned my addr 172.21.15.52:0/4162713686 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bde7fc640 1 --2- 172.21.15.52:0/4162713686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 0x7f7be00af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bdf7fe640 1 -- 172.21.15.52:0/4162713686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 msgr2=0x7f7be00af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bdeffd640 1 --2- 172.21.15.52:0/4162713686 >> v2:172.21.15.119:3300/0 conn(0x7f7be01576a0 0x7f7be00add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bdf7fe640 1 --2- 172.21.15.52:0/4162713686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 0x7f7be00af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bdf7fe640 1 -- 172.21.15.52:0/4162713686 >> v2:172.21.15.119:3300/0 conn(0x7f7be01576a0 msgr2=0x7f7be00add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:11.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bdf7fe640 1 --2- 172.21.15.52:0/4162713686 >> v2:172.21.15.119:3300/0 conn(0x7f7be01576a0 0x7f7be00add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.106+0000 7f7bdf7fe640 1 -- 172.21.15.52:0/4162713686 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be0161210 con 0x7f7be015e1d0 2024-05-11T01:01:11.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.107+0000 7f7bdf7fe640 1 --2- 172.21.15.52:0/4162713686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be015e1d0 0x7f7be00afe90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7be0069230 tx=0x7f7bd802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d822df91527f82c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.107+0000 7f7bddffb640 1 -- 172.21.15.52:0/4162713686 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8036070 con 0x7f7be015e1d0 2024-05-11T01:01:11.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.107+0000 7f7bddffb640 1 -- 172.21.15.52:0/4162713686 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd8031600 con 0x7f7be015e1d0 2024-05-11T01:01:11.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.107+0000 7f7bddffb640 1 -- 172.21.15.52:0/4162713686 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80319d0 con 0x7f7be015e1d0 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.108+0000 7f7be59dd640 1 -- 172.21.15.52:0/4162713686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be015e1d0 msgr2=0x7f7be00afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.108+0000 7f7be59dd640 1 --2- 172.21.15.52:0/4162713686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be015e1d0 0x7f7be00afe90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7be0069230 tx=0x7f7bd802a6a0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.108+0000 7f7be59dd640 1 -- 172.21.15.52:0/4162713686 shutdown_connections 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.108+0000 7f7be59dd640 1 --2- 172.21.15.52:0/4162713686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be015e1d0 0x7f7be00afe90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.108+0000 7f7be59dd640 1 --2- 172.21.15.52:0/4162713686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 0x7f7be00af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.108+0000 7f7be59dd640 1 --2- 172.21.15.52:0/4162713686 >> v2:172.21.15.119:3300/0 conn(0x7f7be01576a0 0x7f7be00add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.108+0000 7f7be59dd640 1 -- 172.21.15.52:0/4162713686 >> 172.21.15.52:0/4162713686 conn(0x7f7be00b2a00 msgr2=0x7f7be00b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.108+0000 7f7be59dd640 1 -- 172.21.15.52:0/4162713686 shutdown_connections 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.108+0000 7f7be59dd640 1 -- 172.21.15.52:0/4162713686 wait complete. 2024-05-11T01:01:11.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.109+0000 7f7be59dd640 1 Processor -- start 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.109+0000 7f7be59dd640 1 -- start start 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7be59dd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 0x7f7be009dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7be59dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be01576a0 0x7f7be01f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7be59dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7be015e1d0 0x7f7be0154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7be59dd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00721b0 con 0x7f7be01576a0 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7be59dd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7be006e050 con 0x7f7be015e1d0 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7be59dd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7be0072330 con 0x7f7be00ae290 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bde7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be01576a0 0x7f7be01f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bde7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be01576a0 0x7f7be01f2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60452/0 (socket says 172.21.15.52:60452) 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bde7fc640 1 -- 172.21.15.52:0/1918310837 learned_addr learned my addr 172.21.15.52:0/1918310837 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bdeffd640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 0x7f7be009dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bdf7fe640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7be015e1d0 0x7f7be0154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bde7fc640 1 -- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 msgr2=0x7f7be009dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bde7fc640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 0x7f7be009dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bde7fc640 1 -- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7be015e1d0 msgr2=0x7f7be0154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bde7fc640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7be015e1d0 0x7f7be0154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.110+0000 7f7bde7fc640 1 -- 172.21.15.52:0/1918310837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be01f2dc0 con 0x7f7be01576a0 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.111+0000 7f7bde7fc640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be01576a0 0x7f7be01f2650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c820 tx=0x7f7bc800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.111+0000 7f7bd7fff640 1 -- 172.21.15.52:0/1918310837 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8015070 con 0x7f7be01576a0 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.111+0000 7f7bd7fff640 1 -- 172.21.15.52:0/1918310837 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc8004220 con 0x7f7be01576a0 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.111+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be01f30a0 con 0x7f7be01576a0 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.111+0000 7f7bd7fff640 1 -- 172.21.15.52:0/1918310837 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800ce10 con 0x7f7be01576a0 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.111+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be01f5e20 con 0x7f7be01576a0 2024-05-11T01:01:11.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.112+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be01f5c40 con 0x7f7be01576a0 2024-05-11T01:01:11.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.112+0000 7f7bd7fff640 1 -- 172.21.15.52:0/1918310837 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7bc8007830 con 0x7f7be01576a0 2024-05-11T01:01:11.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.113+0000 7f7bd7fff640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7bc00762d0 0x7f7bc0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.113+0000 7f7bd7fff640 1 -- 172.21.15.52:0/1918310837 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7bc8097490 con 0x7f7be01576a0 2024-05-11T01:01:11.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.113+0000 7f7bdeffd640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7bc00762d0 0x7f7bc0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.114+0000 7f7bdeffd640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7bc00762d0 0x7f7bc0078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006fd0 tx=0x7f7bd0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.117+0000 7f7bd7fff640 1 -- 172.21.15.52:0/1918310837 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7be01f5c40 con 0x7f7be01576a0 2024-05-11T01:01:11.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.310+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7be0069780 con 0x7f7be01576a0 2024-05-11T01:01:11.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.311+0000 7f7bd7fff640 1 -- 172.21.15.52:0/1918310837 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f7bc8060d60 con 0x7f7be01576a0 2024-05-11T01:01:11.313 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:11.313 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":20,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:02.673511+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:00:59.086890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:00.008780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:11.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7bc00762d0 msgr2=0x7f7bc0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7bc00762d0 0x7f7bc0078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006fd0 tx=0x7f7bd0008040 comp rx=0 tx=0).stop 2024-05-11T01:01:11.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be01576a0 msgr2=0x7f7be01f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be01576a0 0x7f7be01f2650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c820 tx=0x7f7bc800cce0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 shutdown_connections 2024-05-11T01:01:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7bc00762d0 0x7f7bc0078790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7be015e1d0 0x7f7be0154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7be01576a0 0x7f7be01f2650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 --2- 172.21.15.52:0/1918310837 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7be00ae290 0x7f7be009dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 >> 172.21.15.52:0/1918310837 conn(0x7f7be00b2a00 msgr2=0x7f7be0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 shutdown_connections 2024-05-11T01:01:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.314+0000 7f7be59dd640 1 -- 172.21.15.52:0/1918310837 wait complete. 2024-05-11T01:01:11.327 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-11T01:01:11.327 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:11.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.430+0000 7f0882cda640 1 Processor -- start 2024-05-11T01:01:11.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.430+0000 7f0882cda640 1 -- start start 2024-05-11T01:01:11.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f0882cda640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f087c0730d0 0x7f087c06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f0882cda640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f087c06f5f0 con 0x7f087c15c8b0 2024-05-11T01:01:11.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f0882cda640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f087c06f8d0 con 0x7f087c15b4c0 2024-05-11T01:01:11.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f0882cda640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f087c06fbb0 con 0x7f087c0730d0 2024-05-11T01:01:11.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f087bfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f087c15c8b0 0x7f087c06e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44676/0 (socket says 172.21.15.52:44676) 2024-05-11T01:01:11.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f087bfff640 1 -- 172.21.15.52:0/3003344223 learned_addr learned my addr 172.21.15.52:0/3003344223 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:11.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f0881277640 1 --2- 172.21.15.52:0/3003344223 >> v2:172.21.15.119:3300/0 conn(0x7f087c0730d0 0x7f087c06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f087b7fe640 1 -- 172.21.15.52:0/3003344223 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079670700 0 0) 0x7f087c06f5f0 con 0x7f087c15c8b0 2024-05-11T01:01:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f087b7fe640 1 -- 172.21.15.52:0/3003344223 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0868004630 con 0x7f087c15c8b0 2024-05-11T01:01:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f087b7fe640 1 -- 172.21.15.52:0/3003344223 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1517702018 0 0) 0x7f087c06f8d0 con 0x7f087c15b4c0 2024-05-11T01:01:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f087b7fe640 1 -- 172.21.15.52:0/3003344223 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f087c06f5f0 con 0x7f087c15b4c0 2024-05-11T01:01:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f0881277640 1 -- 172.21.15.52:0/3003344223 >> v1:172.21.15.119:6789/0 conn(0x7f087c15b4c0 legacy=0x7f087c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f0881277640 1 -- 172.21.15.52:0/3003344223 >> v1:172.21.15.52:6789/0 conn(0x7f087c15c8b0 legacy=0x7f087c06e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.431+0000 7f0881277640 1 -- 172.21.15.52:0/3003344223 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087c06fe90 con 0x7f087c0730d0 2024-05-11T01:01:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.432+0000 7f0881277640 1 --2- 172.21.15.52:0/3003344223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c0730d0 0x7f087c06eff0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0874009d90 tx=0x7f087402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8726c5e8e87dc687 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.432+0000 7f087b7fe640 1 -- 172.21.15.52:0/3003344223 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874007740 con 0x7f087c0730d0 2024-05-11T01:01:11.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.432+0000 7f087b7fe640 1 -- 172.21.15.52:0/3003344223 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08740078e0 con 0x7f087c0730d0 2024-05-11T01:01:11.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.432+0000 7f087b7fe640 1 -- 172.21.15.52:0/3003344223 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874007c90 con 0x7f087c0730d0 2024-05-11T01:01:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.433+0000 7f0882cda640 1 -- 172.21.15.52:0/3003344223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c0730d0 msgr2=0x7f087c06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.433+0000 7f0882cda640 1 --2- 172.21.15.52:0/3003344223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c0730d0 0x7f087c06eff0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0874009d90 tx=0x7f087402a6f0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.433+0000 7f0882cda640 1 -- 172.21.15.52:0/3003344223 shutdown_connections 2024-05-11T01:01:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.433+0000 7f0882cda640 1 --2- 172.21.15.52:0/3003344223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c0730d0 0x7f087c06eff0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.433+0000 7f0882cda640 1 -- 172.21.15.52:0/3003344223 >> 172.21.15.52:0/3003344223 conn(0x7f087c0b2a00 msgr2=0x7f087c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.433+0000 7f0882cda640 1 -- 172.21.15.52:0/3003344223 shutdown_connections 2024-05-11T01:01:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.433+0000 7f0882cda640 1 -- 172.21.15.52:0/3003344223 wait complete. 2024-05-11T01:01:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.434+0000 7f0882cda640 1 Processor -- start 2024-05-11T01:01:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.434+0000 7f0882cda640 1 -- start start 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0882cda640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f087c0730d0 0x7f087c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0882cda640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f087c15b4c0 0x7f087c09df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0882cda640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c15c8b0 0x7f087c0d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0882cda640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f087c06e050 con 0x7f087c0730d0 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0882cda640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f087c0b5320 con 0x7f087c15c8b0 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0882cda640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f087c06e1d0 con 0x7f087c15b4c0 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0880a76640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f087c0730d0 0x7f087c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0880a76640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f087c0730d0 0x7f087c159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60458/0 (socket says 172.21.15.52:60458) 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0880a76640 1 -- 172.21.15.52:0/3803829364 learned_addr learned my addr 172.21.15.52:0/3803829364 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f087bfff640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f087c15b4c0 0x7f087c09df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0881277640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c15c8b0 0x7f087c0d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0880a76640 1 -- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f087c15b4c0 msgr2=0x7f087c09df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0880a76640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f087c15b4c0 0x7f087c09df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0880a76640 1 -- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c15c8b0 msgr2=0x7f087c0d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0880a76640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c15c8b0 0x7f087c0d0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0880a76640 1 -- 172.21.15.52:0/3803829364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087c09e8e0 con 0x7f087c0730d0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0881277640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c15c8b0 0x7f087c0d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.435+0000 7f0880a76640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f087c0730d0 0x7f087c159980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c490 tx=0x7f086c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.436+0000 7f08797fa640 1 -- 172.21.15.52:0/3803829364 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0052d0 con 0x7f087c0730d0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.436+0000 7f08797fa640 1 -- 172.21.15.52:0/3803829364 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f086c005490 con 0x7f087c0730d0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.436+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087c1f0d90 con 0x7f087c0730d0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.436+0000 7f08797fa640 1 -- 172.21.15.52:0/3803829364 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0108e0 con 0x7f087c0730d0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.436+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087c1f1350 con 0x7f087c0730d0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.437+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f087c069780 con 0x7f087c0730d0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.437+0000 7f08797fa640 1 -- 172.21.15.52:0/3803829364 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f086c010a80 con 0x7f087c0730d0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.438+0000 7f08797fa640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f085c0762d0 0x7f085c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.438+0000 7f08797fa640 1 -- 172.21.15.52:0/3803829364 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f086c099530 con 0x7f087c0730d0 2024-05-11T01:01:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.438+0000 7f087bfff640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f085c0762d0 0x7f085c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.438+0000 7f087bfff640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f085c0762d0 0x7f085c078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f087c0cfde0 tx=0x7f0864009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.442+0000 7f08797fa640 1 -- 172.21.15.52:0/3803829364 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f086c062e90 con 0x7f087c0730d0 2024-05-11T01:01:11.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.635+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f087c15eed0 con 0x7f087c0730d0 2024-05-11T01:01:11.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.636+0000 7f08797fa640 1 -- 172.21.15.52:0/3803829364 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f086c062790 con 0x7f087c0730d0 2024-05-11T01:01:11.637 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:11.637 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":20,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:02.673511+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:00:59.086890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:00.008780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:11.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f085c0762d0 msgr2=0x7f085c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f085c0762d0 0x7f085c078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f087c0cfde0 tx=0x7f0864009040 comp rx=0 tx=0).stop 2024-05-11T01:01:11.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f087c0730d0 msgr2=0x7f087c159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f087c0730d0 0x7f087c159980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c490 tx=0x7f086c00c950 comp rx=0 tx=0).stop 2024-05-11T01:01:11.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 shutdown_connections 2024-05-11T01:01:11.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f085c0762d0 0x7f085c078790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f087c15c8b0 0x7f087c0d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f087c15b4c0 0x7f087c09df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 --2- 172.21.15.52:0/3803829364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f087c0730d0 0x7f087c159980 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 >> 172.21.15.52:0/3803829364 conn(0x7f087c0b2a00 msgr2=0x7f087c15aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 shutdown_connections 2024-05-11T01:01:11.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.638+0000 7f0882cda640 1 -- 172.21.15.52:0/3803829364 wait complete. 2024-05-11T01:01:11.650 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-11T01:01:11.651 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-11T01:01:11.651 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-11T01:01:11.651 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-11T01:01:11.652 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-11T01:01:11.652 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-11T01:01:11.653 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-11T01:01:11.653 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-11T01:01:11.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.832+0000 7fe42e6d0640 1 Processor -- start 2024-05-11T01:01:11.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.832+0000 7fe42e6d0640 1 -- start start 2024-05-11T01:01:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.833+0000 7fe42e6d0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe420050190 0x7fe420054260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.833+0000 7fe42e6d0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4200df380 0x7fe4200382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.833+0000 7fe42e6d0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe420006ae0 con 0x7fe42000aa90 2024-05-11T01:01:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.833+0000 7fe42e6d0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe420006890 con 0x7fe420050190 2024-05-11T01:01:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.833+0000 7fe42e6d0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe420006dc0 con 0x7fe4200df380 2024-05-11T01:01:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.833+0000 7fce483c9640 1 Processor -- start 2024-05-11T01:01:11.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fe42cecd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe420050190 0x7fe420054260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fe42cecd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe420050190 0x7fe420054260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35010/0 (socket says 172.21.15.52:35010) 2024-05-11T01:01:11.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fce483c9640 1 -- start start 2024-05-11T01:01:11.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fe42cecd640 1 -- 172.21.15.52:0/2998288577 learned_addr learned my addr 172.21.15.52:0/2998288577 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fe42cecd640 1 -- 172.21.15.52:0/2998288577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4200df380 msgr2=0x7fe4200382b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T01:01:11.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fe42cecd640 1 --2- 172.21.15.52:0/2998288577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4200df380 0x7fe4200382b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fe42cecd640 1 -- 172.21.15.52:0/2998288577 >> v1:172.21.15.119:6789/0 conn(0x7fe42000aa90 legacy=0x7fe42004fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fe42cecd640 1 -- 172.21.15.52:0/2998288577 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4200070a0 con 0x7fe420050190 2024-05-11T01:01:11.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fbc535da640 1 Processor -- start 2024-05-11T01:01:11.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.834+0000 7fcda98d1640 1 Processor -- start 2024-05-11T01:01:11.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.835+0000 7fecd09ca640 1 Processor -- start 2024-05-11T01:01:11.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.835+0000 7fecd09ca640 1 -- start start 2024-05-11T01:01:11.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.835+0000 7fbc535da640 1 -- start start 2024-05-11T01:01:11.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.835+0000 7fe42cecd640 1 --2- 172.21.15.52:0/2998288577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe420050190 0x7fe420054260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe42807bb40 tx=0x7fe42806a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205833b5fbf57770 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7feccb7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fecc40df210 0x7fecc403a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58046/0 (socket says 172.21.15.52:58046) 2024-05-11T01:01:11.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7feccb7fe640 1 -- 172.21.15.52:0/517452095 learned_addr learned my addr 172.21.15.52:0/517452095 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:11.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.835+0000 7fce483c9640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fce40158f50 0x7fce4015f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fce483c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce400cdee0 0x7fce400add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fce483c9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce40159330 con 0x7fce4015b950 2024-05-11T01:01:11.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fce483c9640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fce4015c3a0 con 0x7fce40158f50 2024-05-11T01:01:11.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fce483c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fce4015c680 con 0x7fce400cdee0 2024-05-11T01:01:11.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fce45964640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fce4015b950 0x7fce400cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:52854/0 (socket says 172.21.15.52:52854) 2024-05-11T01:01:11.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fce45964640 1 -- 172.21.15.52:0/598718601 learned_addr learned my addr 172.21.15.52:0/598718601 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:11.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.835+0000 7ff266d32640 1 Processor -- start 2024-05-11T01:01:11.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.835+0000 7ff266d32640 1 -- start start 2024-05-11T01:01:11.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fbc535da640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc440061f0 0x7fbc44056fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fbc535da640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc440075e0 0x7fbc4403c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fbc535da640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc44057a60 con 0x7fbc440574e0 2024-05-11T01:01:11.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc535da640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fbc44057d40 con 0x7fbc440075e0 2024-05-11T01:01:11.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc535da640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbc44058020 con 0x7fbc440061f0 2024-05-11T01:01:11.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fbc50b75640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbc440574e0 0x7fbc4403c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:52866/0 (socket says 172.21.15.52:52866) 2024-05-11T01:01:11.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc50b75640 1 -- 172.21.15.52:0/360541412 learned_addr learned my addr 172.21.15.52:0/360541412 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:11.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc51b77640 1 --2- 172.21.15.52:0/360541412 >> v2:172.21.15.52:3300/0 conn(0x7fbc440075e0 0x7fbc4403c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc51376640 1 --2- 172.21.15.52:0/360541412 >> v2:172.21.15.119:3300/0 conn(0x7fbc440061f0 0x7fbc44056fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7ff266d32640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2600cb320 0x7ff2600ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fecd09ca640 1 --2- 172.21.15.52:0/517452095 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e31d0 0x7fecc40071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fce46966640 1 --2- 172.21.15.52:0/598718601 >> v2:172.21.15.119:3300/0 conn(0x7fce400cdee0 0x7fce400add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fecd09ca640 1 -- 172.21.15.52:0/517452095 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc4054630 con 0x7fecc40df210 2024-05-11T01:01:11.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fecd09ca640 1 -- 172.21.15.52:0/517452095 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc4054910 con 0x7fecc40e2d70 2024-05-11T01:01:11.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.835+0000 7fcda98d1640 1 -- start start 2024-05-11T01:01:11.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fcda98d1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda40b4900 0x7fcda4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fcda98d1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcda406ea60 0x7fcda4163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fecd09ca640 1 -- 172.21.15.52:0/517452095 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fecc40543e0 con 0x7fecc40e31d0 2024-05-11T01:01:11.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fcda98d1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcda4070890 con 0x7fcda4163a60 2024-05-11T01:01:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fcda98d1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fcda406f890 con 0x7fcda406ea60 2024-05-11T01:01:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fcda98d1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fcda4164590 con 0x7fcda40b4900 2024-05-11T01:01:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fcda2ffd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda40b4900 0x7fcda4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7ff266d32640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff26015c170 0x7ff2600cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc51b77640 1 -- 172.21.15.52:0/360541412 >> v2:172.21.15.119:3300/0 conn(0x7fbc440061f0 msgr2=0x7fbc44056fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:11.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc51b77640 1 --2- 172.21.15.52:0/360541412 >> v2:172.21.15.119:3300/0 conn(0x7fbc440061f0 0x7fbc44056fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc51b77640 1 -- 172.21.15.52:0/360541412 >> v1:172.21.15.119:6789/0 conn(0x7fbc440574e0 legacy=0x7fbc4403c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc43fff640 1 -- 172.21.15.52:0/360541412 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1289047467 0 0) 0x7fbc44057a60 con 0x7fbc440574e0 2024-05-11T01:01:11.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fbc51b77640 1 -- 172.21.15.52:0/360541412 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc44058300 con 0x7fbc440075e0 2024-05-11T01:01:11.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fcda27fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcda4163a60 0x7fcda406e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:52880/0 (socket says 172.21.15.52:52880) 2024-05-11T01:01:11.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fcda27fc640 1 -- 172.21.15.52:0/482935188 learned_addr learned my addr 172.21.15.52:0/482935188 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:11.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fce46966640 1 -- 172.21.15.52:0/598718601 >> v2:172.21.15.52:3300/0 conn(0x7fce40158f50 msgr2=0x7fce4015f8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T01:01:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fce46966640 1 --2- 172.21.15.52:0/598718601 >> v2:172.21.15.52:3300/0 conn(0x7fce40158f50 0x7fce4015f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fce46966640 1 -- 172.21.15.52:0/598718601 >> v1:172.21.15.119:6789/0 conn(0x7fce4015b950 legacy=0x7fce400cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7feccbfff640 1 --2- 172.21.15.52:0/517452095 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e31d0 0x7fecc40071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fce46966640 1 -- 172.21.15.52:0/598718601 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4015f0e0 con 0x7fce400cdee0 2024-05-11T01:01:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.835+0000 7fe41ffff640 1 -- 172.21.15.52:0/2998288577 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280af070 con 0x7fe420050190 2024-05-11T01:01:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fe41ffff640 1 -- 172.21.15.52:0/2998288577 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4280ab670 con 0x7fe420050190 2024-05-11T01:01:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2998288577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe420050190 msgr2=0x7fe420054260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.836+0000 7fe42e6d0640 1 --2- 172.21.15.52:0/2998288577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe420050190 0x7fe420054260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe42807bb40 tx=0x7fe42806a270 comp rx=0 tx=0).stop 2024-05-11T01:01:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2998288577 shutdown_connections 2024-05-11T01:01:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fe42e6d0640 1 --2- 172.21.15.52:0/2998288577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4200df380 0x7fe4200382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fe42e6d0640 1 --2- 172.21.15.52:0/2998288577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe420050190 0x7fe420054260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2998288577 >> 172.21.15.52:0/2998288577 conn(0x7fe420009fa0 msgr2=0x7fe42000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2998288577 shutdown_connections 2024-05-11T01:01:11.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2998288577 wait complete. 2024-05-11T01:01:11.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7fecca7fc640 1 -- 172.21.15.52:0/517452095 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776140319 0 0) 0x7fecc4054630 con 0x7fecc40df210 2024-05-11T01:01:11.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7fecca7fc640 1 -- 172.21.15.52:0/517452095 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecbc003520 con 0x7fecc40df210 2024-05-11T01:01:11.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7f4ca9de3640 1 Processor -- start 2024-05-11T01:01:11.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.837+0000 7fcda37fe640 1 --2- 172.21.15.52:0/482935188 >> v2:172.21.15.119:3300/0 conn(0x7fcda406ea60 0x7fcda4163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7f4ca9de3640 1 -- start start 2024-05-11T01:01:11.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7ff266d32640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 0x7ff26015f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7ff266d32640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff2600cbc40 con 0x7ff26015c170 2024-05-11T01:01:11.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7ff25ffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff26015c170 0x7ff2600cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7ff25ffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff26015c170 0x7ff2600cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35028/0 (socket says 172.21.15.52:35028) 2024-05-11T01:01:11.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7feccbfff640 1 -- 172.21.15.52:0/517452095 >> v1:172.21.15.119:6789/0 conn(0x7fecc40e2d70 legacy=0x7fecc4006ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T01:01:11.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7ff25ffff640 1 -- 172.21.15.52:0/708549 learned_addr learned my addr 172.21.15.52:0/708549 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7fbc51b77640 1 --2- 172.21.15.52:0/360541412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc440075e0 0x7fbc4403c710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0521a0 tx=0x7fbc4c06a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cb4d1017373165 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7ff2652cf640 1 --2- 172.21.15.52:0/708549 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 0x7ff26015f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7fbc43fff640 1 -- 172.21.15.52:0/360541412 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0b2070 con 0x7fbc440075e0 2024-05-11T01:01:11.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7ff266d32640 1 -- 172.21.15.52:0/708549 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff2600cf090 con 0x7ff2600cb320 2024-05-11T01:01:11.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7ff266d32640 1 -- 172.21.15.52:0/708549 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff2600cf370 con 0x7ff26015ece0 2024-05-11T01:01:11.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fbc43fff640 1 -- 172.21.15.52:0/360541412 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0ab7d0 con 0x7fbc440075e0 2024-05-11T01:01:11.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fce46966640 1 --2- 172.21.15.52:0/598718601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce400cdee0 0x7fce400add50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce400cb340 tx=0x7fce3802c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ac429339dcae8bc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fbc43fff640 1 -- 172.21.15.52:0/360541412 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0abc40 con 0x7fbc440075e0 2024-05-11T01:01:11.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7feccbfff640 1 -- 172.21.15.52:0/517452095 >> v1:172.21.15.52:6789/0 conn(0x7fecc40df210 legacy=0x7fecc403a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.838+0000 7feccbfff640 1 -- 172.21.15.52:0/517452095 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc4054bf0 con 0x7fecc40e31d0 2024-05-11T01:01:11.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fcda1ffb640 1 -- 172.21.15.52:0/482935188 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398802593 0 0) 0x7fcda4070890 con 0x7fcda4163a60 2024-05-11T01:01:11.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fcda1ffb640 1 -- 172.21.15.52:0/482935188 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd90003520 con 0x7fcda4163a60 2024-05-11T01:01:11.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7feccbfff640 1 --2- 172.21.15.52:0/517452095 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e31d0 0x7fecc40071e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feccc06b4c0 tx=0x7feccc06a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254e53a55d724642 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fcda1ffb640 1 -- 172.21.15.52:0/482935188 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 453268005 0 0) 0x7fcd90003520 con 0x7fcda4163a60 2024-05-11T01:01:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7ff264ace640 1 --2- 172.21.15.52:0/708549 >> v2:172.21.15.119:3300/0 conn(0x7ff2600cb320 0x7ff2600ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fcda37fe640 1 -- 172.21.15.52:0/482935188 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda40b4900 msgr2=0x7fcda4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fcda37fe640 1 --2- 172.21.15.52:0/482935188 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda40b4900 0x7fcda4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fcda37fe640 1 -- 172.21.15.52:0/482935188 >> v1:172.21.15.119:6789/0 conn(0x7fcda4163a60 legacy=0x7fcda406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fcda37fe640 1 -- 172.21.15.52:0/482935188 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda4164870 con 0x7fcda406ea60 2024-05-11T01:01:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fcda2ffd640 1 --2- 172.21.15.52:0/482935188 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda40b4900 0x7fcda4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7ff2652cf640 1 -- 172.21.15.52:0/708549 >> v2:172.21.15.119:3300/0 conn(0x7ff2600cb320 msgr2=0x7ff2600ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7ff2652cf640 1 --2- 172.21.15.52:0/708549 >> v2:172.21.15.119:3300/0 conn(0x7ff2600cb320 0x7ff2600ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7ff2652cf640 1 -- 172.21.15.52:0/708549 >> v2:172.21.15.52:3300/0 conn(0x7ff26015c170 msgr2=0x7ff2600cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7ff2652cf640 1 --2- 172.21.15.52:0/708549 >> v2:172.21.15.52:3300/0 conn(0x7ff26015c170 0x7ff2600cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7ff2652cf640 1 -- 172.21.15.52:0/708549 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2600cf650 con 0x7ff26015ece0 2024-05-11T01:01:11.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fe42e6d0640 1 Processor -- start 2024-05-11T01:01:11.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.839+0000 7fe42e6d0640 1 -- start start 2024-05-11T01:01:11.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fbc535da640 1 -- 172.21.15.52:0/360541412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc440075e0 msgr2=0x7fbc4403c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7f4ca8de1640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f4ca40cd4a0 0x7f4ca40cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58058/0 (socket says 172.21.15.52:58058) 2024-05-11T01:01:11.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7f4ca8de1640 1 -- 172.21.15.52:0/4098259806 learned_addr learned my addr 172.21.15.52:0/4098259806 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fbc535da640 1 --2- 172.21.15.52:0/360541412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc440075e0 0x7fbc4403c710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0521a0 tx=0x7fbc4c06a270 comp rx=0 tx=0).stop 2024-05-11T01:01:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7f4ca9de3640 1 --2- 172.21.15.52:0/4098259806 >> v2:172.21.15.52:3300/0 conn(0x7f4ca415c450 0x7f4ca40cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7f4ca9de3640 1 --2- 172.21.15.52:0/4098259806 >> v2:172.21.15.119:3300/0 conn(0x7f4ca406eb90 0x7f4ca40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7f4ca95e2640 1 --2- 172.21.15.52:0/4098259806 >> v2:172.21.15.119:3300/0 conn(0x7f4ca406eb90 0x7f4ca40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7f4ca9de3640 1 -- 172.21.15.52:0/4098259806 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca40cbba0 con 0x7f4ca40cd4a0 2024-05-11T01:01:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7f4ca9de3640 1 -- 172.21.15.52:0/4098259806 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca40cbe80 con 0x7f4ca415c450 2024-05-11T01:01:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fbc535da640 1 -- 172.21.15.52:0/360541412 shutdown_connections 2024-05-11T01:01:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fbc535da640 1 --2- 172.21.15.52:0/360541412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc440075e0 0x7fbc4403c710 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fbc535da640 1 --2- 172.21.15.52:0/360541412 >> v2:172.21.15.119:3300/0 conn(0x7fbc440061f0 0x7fbc44056fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fbc535da640 1 -- 172.21.15.52:0/360541412 >> 172.21.15.52:0/360541412 conn(0x7fbc44058ed0 msgr2=0x7fbc440592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fecca7fc640 1 -- 172.21.15.52:0/517452095 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc0b2070 con 0x7fecc40e31d0 2024-05-11T01:01:11.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fecca7fc640 1 -- 172.21.15.52:0/517452095 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feccc0ae970 con 0x7fecc40e31d0 2024-05-11T01:01:11.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fecca7fc640 1 -- 172.21.15.52:0/517452095 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc0aede0 con 0x7fecc40e31d0 2024-05-11T01:01:11.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fce45163640 1 -- 172.21.15.52:0/598718601 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38033740 con 0x7fce400cdee0 2024-05-11T01:01:11.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fcda37fe640 1 --2- 172.21.15.52:0/482935188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda406ea60 0x7fcda4163ec0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009d90 tx=0x7fcd9402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc55a589c47b4cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fce45163640 1 -- 172.21.15.52:0/598718601 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce380338e0 con 0x7fce400cdee0 2024-05-11T01:01:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7fce45163640 1 -- 172.21.15.52:0/598718601 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38033cd0 con 0x7fce400cdee0 2024-05-11T01:01:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.840+0000 7f4ca3fff640 1 --2- 172.21.15.52:0/4098259806 >> v2:172.21.15.52:3300/0 conn(0x7f4ca415c450 0x7f4ca40cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7f4ca37fe640 1 -- 172.21.15.52:0/4098259806 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101726748 0 0) 0x7f4ca40cbba0 con 0x7f4ca40cd4a0 2024-05-11T01:01:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7ff2652cf640 1 --2- 172.21.15.52:0/708549 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 0x7ff26015f130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff26006d570 tx=0x7ff25802c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab2a30b61121f35f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42e6d0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe42000aa90 0x7fe42004d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7f4ca9de3640 1 -- 172.21.15.52:0/4098259806 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca406e0b0 con 0x7f4ca406eb90 2024-05-11T01:01:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fcda1ffb640 1 -- 172.21.15.52:0/482935188 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c0016f0 con 0x7fcda406ea60 2024-05-11T01:01:11.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fecd09ca640 1 -- 172.21.15.52:0/517452095 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e31d0 msgr2=0x7fecc40071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fecd09ca640 1 --2- 172.21.15.52:0/517452095 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e31d0 0x7fecc40071e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feccc06b4c0 tx=0x7feccc06a270 comp rx=0 tx=0).stop 2024-05-11T01:01:11.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fecd09ca640 1 -- 172.21.15.52:0/517452095 shutdown_connections 2024-05-11T01:01:11.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7ff25f7fe640 1 -- 172.21.15.52:0/708549 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258007740 con 0x7ff26015ece0 2024-05-11T01:01:11.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fcda1ffb640 1 -- 172.21.15.52:0/482935188 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd94007540 con 0x7fcda406ea60 2024-05-11T01:01:11.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7f4ca37fe640 1 -- 172.21.15.52:0/4098259806 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c94003520 con 0x7f4ca40cd4a0 2024-05-11T01:01:11.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fecd09ca640 1 --2- 172.21.15.52:0/517452095 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e31d0 0x7fecc40071e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7ff25f7fe640 1 -- 172.21.15.52:0/708549 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2580078e0 con 0x7ff26015ece0 2024-05-11T01:01:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fecd09ca640 1 -- 172.21.15.52:0/517452095 >> 172.21.15.52:0/517452095 conn(0x7fecc400a3f0 msgr2=0x7fecc400a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7ff25f7fe640 1 -- 172.21.15.52:0/708549 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258007740 con 0x7ff26015ece0 2024-05-11T01:01:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fbc535da640 1 -- 172.21.15.52:0/360541412 shutdown_connections 2024-05-11T01:01:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fbc535da640 1 -- 172.21.15.52:0/360541412 wait complete. 2024-05-11T01:01:11.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fecd09ca640 1 -- 172.21.15.52:0/517452095 shutdown_connections 2024-05-11T01:01:11.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fecd09ca640 1 -- 172.21.15.52:0/517452095 wait complete. 2024-05-11T01:01:11.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7f4ca3fff640 1 -- 172.21.15.52:0/4098259806 >> v2:172.21.15.119:3300/0 conn(0x7f4ca406eb90 msgr2=0x7f4ca40cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:11.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7f4ca3fff640 1 --2- 172.21.15.52:0/4098259806 >> v2:172.21.15.119:3300/0 conn(0x7f4ca406eb90 0x7f4ca40cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7f4ca3fff640 1 -- 172.21.15.52:0/4098259806 >> v1:172.21.15.52:6789/0 conn(0x7f4ca40cd4a0 legacy=0x7f4ca40cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7f4ca3fff640 1 -- 172.21.15.52:0/4098259806 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca406e390 con 0x7f4ca415c450 2024-05-11T01:01:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42d6ce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe42000aa90 0x7fe42004d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42d6ce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe42000aa90 0x7fe42004d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35050/0 (socket says 172.21.15.52:35050) 2024-05-11T01:01:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42d6ce640 1 -- 172.21.15.52:0/2031764674 learned_addr learned my addr 172.21.15.52:0/2031764674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42e6d0640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe420050190 0x7fe420048730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42cecd640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe420050190 0x7fe420048730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42e6d0640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4200df380 0x7fe4200ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe420052b30 con 0x7fe42000aa90 2024-05-11T01:01:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe4200547a0 con 0x7fe4200df380 2024-05-11T01:01:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe420052cb0 con 0x7fe420050190 2024-05-11T01:01:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fe42decf640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4200df380 0x7fe4200ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fcda1ffb640 1 -- 172.21.15.52:0/482935188 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94007910 con 0x7fcda406ea60 2024-05-11T01:01:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fcda98d1640 1 -- 172.21.15.52:0/482935188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda406ea60 msgr2=0x7fcda4163ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fcda98d1640 1 --2- 172.21.15.52:0/482935188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda406ea60 0x7fcda4163ec0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009d90 tx=0x7fcd9402a5f0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fce483c9640 1 -- 172.21.15.52:0/598718601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce400cdee0 msgr2=0x7fce400add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.841+0000 7fce483c9640 1 --2- 172.21.15.52:0/598718601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce400cdee0 0x7fce400add50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce400cb340 tx=0x7fce3802c990 comp rx=0 tx=0).stop 2024-05-11T01:01:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fce483c9640 1 -- 172.21.15.52:0/598718601 shutdown_connections 2024-05-11T01:01:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fce483c9640 1 --2- 172.21.15.52:0/598718601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce400cdee0 0x7fce400add50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fce483c9640 1 --2- 172.21.15.52:0/598718601 >> v2:172.21.15.52:3300/0 conn(0x7fce40158f50 0x7fce4015f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fce483c9640 1 -- 172.21.15.52:0/598718601 >> 172.21.15.52:0/598718601 conn(0x7fce400b2900 msgr2=0x7fce400b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fe42d6ce640 1 -- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe420050190 msgr2=0x7fe420048730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fe42d6ce640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe420050190 0x7fe420048730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fe42d6ce640 1 -- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4200df380 msgr2=0x7fe4200ef5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fe42d6ce640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4200df380 0x7fe4200ef5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fe42d6ce640 1 -- 172.21.15.52:0/2031764674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42004e540 con 0x7fe42000aa90 2024-05-11T01:01:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fe42decf640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4200df380 0x7fe4200ef5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fe42cecd640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe420050190 0x7fe420048730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fce483c9640 1 -- 172.21.15.52:0/598718601 shutdown_connections 2024-05-11T01:01:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7ff266d32640 1 -- 172.21.15.52:0/708549 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 msgr2=0x7ff26015f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fe42d6ce640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe42000aa90 0x7fe42004d5e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c500 tx=0x7fe42400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fce483c9640 1 -- 172.21.15.52:0/598718601 wait complete. 2024-05-11T01:01:11.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7ff266d32640 1 --2- 172.21.15.52:0/708549 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 0x7ff26015f130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff26006d570 tx=0x7ff25802c760 comp rx=0 tx=0).stop 2024-05-11T01:01:11.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fdfa4122640 1 Processor -- start 2024-05-11T01:01:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fdfa4122640 1 -- start start 2024-05-11T01:01:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7ff266d32640 1 -- 172.21.15.52:0/708549 shutdown_connections 2024-05-11T01:01:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7ff266d32640 1 --2- 172.21.15.52:0/708549 >> v2:172.21.15.52:3300/0 conn(0x7ff26015c170 0x7ff2600cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fcda98d1640 1 -- 172.21.15.52:0/482935188 shutdown_connections 2024-05-11T01:01:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fcda98d1640 1 --2- 172.21.15.52:0/482935188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda406ea60 0x7fcda4163ec0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.842+0000 7fcda98d1640 1 --2- 172.21.15.52:0/482935188 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda40b4900 0x7fcda4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7ff266d32640 1 --2- 172.21.15.52:0/708549 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 0x7ff26015f130 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7fcda98d1640 1 -- 172.21.15.52:0/482935188 >> 172.21.15.52:0/482935188 conn(0x7fcda40b2d80 msgr2=0x7fcda40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7ff266d32640 1 --2- 172.21.15.52:0/708549 >> v2:172.21.15.119:3300/0 conn(0x7ff2600cb320 0x7ff2600ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7ff266d32640 1 -- 172.21.15.52:0/708549 >> 172.21.15.52:0/708549 conn(0x7ff2600cd750 msgr2=0x7ff2600cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7ff266d32640 1 -- 172.21.15.52:0/708549 shutdown_connections 2024-05-11T01:01:11.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7ff266d32640 1 -- 172.21.15.52:0/708549 wait complete. 2024-05-11T01:01:11.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7fbc535da640 1 Processor -- start 2024-05-11T01:01:11.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7fbc535da640 1 -- start start 2024-05-11T01:01:11.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7fcda98d1640 1 -- 172.21.15.52:0/482935188 shutdown_connections 2024-05-11T01:01:11.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7fcda98d1640 1 -- 172.21.15.52:0/482935188 wait complete. 2024-05-11T01:01:11.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7f4ca3fff640 1 --2- 172.21.15.52:0/4098259806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca415c450 0x7f4ca40cb230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ca415ec60 tx=0x7f4c9002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d8839ff3f34d65 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7fe41dffb640 1 -- 172.21.15.52:0/2031764674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4240052d0 con 0x7fe42000aa90 2024-05-11T01:01:11.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7fe41dffb640 1 -- 172.21.15.52:0/2031764674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe424005470 con 0x7fe42000aa90 2024-05-11T01:01:11.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.843+0000 7fe41dffb640 1 -- 172.21.15.52:0/2031764674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424012870 con 0x7fe42000aa90 2024-05-11T01:01:11.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42004dd00 con 0x7fe42000aa90 2024-05-11T01:01:11.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fdfa4122640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf9c15d190 0x7fdf9c06fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fecd09ca640 1 Processor -- start 2024-05-11T01:01:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fce483c9640 1 Processor -- start 2024-05-11T01:01:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fce483c9640 1 -- start start 2024-05-11T01:01:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42004e260 con 0x7fe42000aa90 2024-05-11T01:01:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc535da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc440061f0 0x7fbc441816a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc535da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc440075e0 0x7fbc4416b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc535da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc440574e0 0x7fbc440e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc535da640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4403edc0 con 0x7fbc440075e0 2024-05-11T01:01:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc535da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbc440e3510 con 0x7fbc440574e0 2024-05-11T01:01:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc535da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbc44007a30 con 0x7fbc440061f0 2024-05-11T01:01:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fecd09ca640 1 -- start start 2024-05-11T01:01:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc51b77640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc440574e0 0x7fbc440e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc51376640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc440061f0 0x7fbc441816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc51b77640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc440574e0 0x7fbc440e5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:46342/0 (socket says 172.21.15.52:46342) 2024-05-11T01:01:11.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fbc51b77640 1 -- 172.21.15.52:0/2626069710 learned_addr learned my addr 172.21.15.52:0/2626069710 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7ff266d32640 1 Processor -- start 2024-05-11T01:01:11.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe420005380 con 0x7fe42000aa90 2024-05-11T01:01:11.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7f4ca37fe640 1 -- 172.21.15.52:0/4098259806 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90037070 con 0x7f4ca415c450 2024-05-11T01:01:11.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7f4ca37fe640 1 -- 172.21.15.52:0/4098259806 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c90032440 con 0x7f4ca415c450 2024-05-11T01:01:11.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fdfa1ebe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf9c15d190 0x7fdf9c06fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.844+0000 7fdfa1ebe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf9c15d190 0x7fdf9c06fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:46354/0 (socket says 172.21.15.52:46354) 2024-05-11T01:01:11.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fdfa1ebe640 1 -- 172.21.15.52:0/4233785798 learned_addr learned my addr 172.21.15.52:0/4233785798 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7ff266d32640 1 -- start start 2024-05-11T01:01:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fce483c9640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fce400cdee0 0x7fce400c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fce483c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce40158f50 0x7fce40154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fce483c9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4015b950 0x7fce400cff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fce483c9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fce400b0400 con 0x7fce4015b950 2024-05-11T01:01:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fce483c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fce400b0280 con 0x7fce40158f50 2024-05-11T01:01:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fce483c9640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fce400b33f0 con 0x7fce400cdee0 2024-05-11T01:01:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fdfa4122640 1 --2- 172.21.15.52:0/4233785798 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cdf10 0x7fdf9c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fdfa4122640 1 -- 172.21.15.52:0/4233785798 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c160f00 con 0x7fdf9c0cce40 2024-05-11T01:01:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fce46165640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fce400cdee0 0x7fce400c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fdfa4122640 1 -- 172.21.15.52:0/4233785798 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c161c80 con 0x7fdf9c15d190 2024-05-11T01:01:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fdfa4122640 1 -- 172.21.15.52:0/4233785798 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c0cd650 con 0x7fdf9c0cdf10 2024-05-11T01:01:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fdfa26bf640 1 --2- 172.21.15.52:0/4233785798 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cdf10 0x7fdf9c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7f4ca9de3640 1 -- 172.21.15.52:0/4098259806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca415c450 msgr2=0x7f4ca40cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7f4ca37fe640 1 -- 172.21.15.52:0/4098259806 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90037070 con 0x7f4ca415c450 2024-05-11T01:01:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7f4ca9de3640 1 --2- 172.21.15.52:0/4098259806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca415c450 0x7f4ca40cb230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ca415ec60 tx=0x7f4c9002a110 comp rx=0 tx=0).stop 2024-05-11T01:01:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7f4ca9de3640 1 -- 172.21.15.52:0/4098259806 shutdown_connections 2024-05-11T01:01:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7f4ca9de3640 1 --2- 172.21.15.52:0/4098259806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca415c450 0x7f4ca40cb230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7f4ca9de3640 1 --2- 172.21.15.52:0/4098259806 >> v2:172.21.15.119:3300/0 conn(0x7f4ca406eb90 0x7f4ca40cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7f4ca9de3640 1 -- 172.21.15.52:0/4098259806 >> 172.21.15.52:0/4098259806 conn(0x7f4ca40b26f0 msgr2=0x7f4ca40b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fce46165640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fce400cdee0 0x7fce400c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:40974/0 (socket says 172.21.15.52:40974) 2024-05-11T01:01:11.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fce46165640 1 -- 172.21.15.52:0/1791624207 learned_addr learned my addr 172.21.15.52:0/1791624207 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fce46966640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4015b950 0x7fce400cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fecd09ca640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecc40df210 0x7fecc4038700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fdfa26bf640 1 -- 172.21.15.52:0/4233785798 >> v2:172.21.15.119:3300/0 conn(0x7fdf9c15d190 msgr2=0x7fdf9c06fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:11.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fdfa26bf640 1 --2- 172.21.15.52:0/4233785798 >> v2:172.21.15.119:3300/0 conn(0x7fdf9c15d190 0x7fdf9c06fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fdfa26bf640 1 -- 172.21.15.52:0/4233785798 >> v1:172.21.15.52:6789/0 conn(0x7fdf9c0cce40 legacy=0x7fdf9c0cd800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T01:01:11.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fecd09ca640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e2d70 0x7fecc4174d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fce45964640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce40158f50 0x7fce40154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fdfa26bf640 1 -- 172.21.15.52:0/4233785798 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c0add50 con 0x7fdf9c0cdf10 2024-05-11T01:01:11.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fbc50b75640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc440075e0 0x7fbc4416b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fbc51376640 1 -- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc440574e0 msgr2=0x7fbc440e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fbc51376640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc440574e0 0x7fbc440e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fbc51376640 1 -- 172.21.15.52:0/2626069710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc440075e0 msgr2=0x7fbc4416b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fbc51376640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc440075e0 0x7fbc4416b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fbc51376640 1 -- 172.21.15.52:0/2626069710 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4416e830 con 0x7fbc440061f0 2024-05-11T01:01:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fbc51376640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc440061f0 0x7fbc441816a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800e850 tx=0x7fbc4800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fbc41ffb640 1 -- 172.21.15.52:0/2626069710 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48017070 con 0x7fbc440061f0 2024-05-11T01:01:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fbc41ffb640 1 -- 172.21.15.52:0/2626069710 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4800c940 con 0x7fbc440061f0 2024-05-11T01:01:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fbc41ffb640 1 -- 172.21.15.52:0/2626069710 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800cd10 con 0x7fbc440061f0 2024-05-11T01:01:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fce46165640 1 -- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce40158f50 msgr2=0x7fce40154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fce46165640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce40158f50 0x7fce40154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fce46165640 1 -- 172.21.15.52:0/1791624207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4015b950 msgr2=0x7fce400cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fce46165640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4015b950 0x7fce400cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7feccb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecc40df210 0x7fecc4038700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fce46165640 1 -- 172.21.15.52:0/1791624207 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce40155190 con 0x7fce400cdee0 2024-05-11T01:01:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7feccb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecc40df210 0x7fecc4038700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35092/0 (socket says 172.21.15.52:35092) 2024-05-11T01:01:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7feccb7fe640 1 -- 172.21.15.52:0/1350531231 learned_addr learned my addr 172.21.15.52:0/1350531231 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fce45964640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce40158f50 0x7fce40154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7feccaffd640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e2d70 0x7fecc4174d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7ff266d32640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2600cb320 0x7ff2600cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fcda98d1640 1 Processor -- start 2024-05-11T01:01:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.845+0000 7fcda98d1640 1 -- start start 2024-05-11T01:01:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda98d1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda406ea60 0x7fcda4205a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda98d1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcda40b4900 0x7fcda42012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda98d1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda4163a60 0x7fcda415bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda98d1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4074550 con 0x7fcda40b4900 2024-05-11T01:01:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda98d1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcda406fe30 con 0x7fcda406ea60 2024-05-11T01:01:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda98d1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fcda40b4ce0 con 0x7fcda4163a60 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda37fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda4163a60 0x7fcda415bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda37fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda4163a60 0x7fcda415bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:40978/0 (socket says 172.21.15.52:40978) 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda37fe640 1 -- 172.21.15.52:0/1657869674 learned_addr learned my addr 172.21.15.52:0/1657869674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.846+0000 7fcda27fc640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcda40b4900 0x7fcda42012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fcda2ffd640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda406ea60 0x7fcda4205a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fcda27fc640 1 -- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda4163a60 msgr2=0x7fcda415bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fcda27fc640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcda4163a60 0x7fcda415bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fcda27fc640 1 -- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda406ea60 msgr2=0x7fcda4205a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fcda27fc640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda406ea60 0x7fcda4205a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fcda27fc640 1 -- 172.21.15.52:0/1657869674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda4205f90 con 0x7fcda40b4900 2024-05-11T01:01:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7ff266d32640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff26015c170 0x7ff2600afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc440e52c0 con 0x7fbc440061f0 2024-05-11T01:01:11.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fce46165640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fce400cdee0 0x7fce400c5b10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fce3000b480 tx=0x7fce3000b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7ff264ace640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2600cb320 0x7ff2600cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7ff264ace640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2600cb320 0x7ff2600cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:46390/0 (socket says 172.21.15.52:46390) 2024-05-11T01:01:11.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7ff264ace640 1 -- 172.21.15.52:0/1277727154 learned_addr learned my addr 172.21.15.52:0/1277727154 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fe41dffb640 1 -- 172.21.15.52:0/2031764674 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe424012a10 con 0x7fe42000aa90 2024-05-11T01:01:11.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7ff266d32640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 0x7ff2601f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7ff25ffff640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff26015c170 0x7ff2600afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fce36ffd640 1 -- 172.21.15.52:0/1791624207 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30014070 con 0x7fce400cdee0 2024-05-11T01:01:11.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff260158fb0 con 0x7ff26015c170 2024-05-11T01:01:11.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fce36ffd640 1 -- 172.21.15.52:0/1791624207 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce30010890 con 0x7fce400cdee0 2024-05-11T01:01:11.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fcda27fc640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcda40b4900 0x7fcda42012e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c700 tx=0x7fcd8c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fce483c9640 1 -- 172.21.15.52:0/1791624207 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce400c61f0 con 0x7fce400cdee0 2024-05-11T01:01:11.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fce483c9640 1 -- 172.21.15.52:0/1791624207 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce400c67b0 con 0x7fce400cdee0 2024-05-11T01:01:11.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7f4ca9de3640 1 -- 172.21.15.52:0/4098259806 shutdown_connections 2024-05-11T01:01:11.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7f4ca9de3640 1 -- 172.21.15.52:0/4098259806 wait complete. 2024-05-11T01:01:11.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fcd9bfff640 1 -- 172.21.15.52:0/1657869674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c005ad0 con 0x7fcda40b4900 2024-05-11T01:01:11.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda40d1110 con 0x7fcda40b4900 2024-05-11T01:01:11.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda40d16d0 con 0x7fcda40b4900 2024-05-11T01:01:11.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fcd9bfff640 1 -- 172.21.15.52:0/1657869674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c005c90 con 0x7fcda40b4900 2024-05-11T01:01:11.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fcd9bfff640 1 -- 172.21.15.52:0/1657869674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c0072b0 con 0x7fcda40b4900 2024-05-11T01:01:11.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.847+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc440e6000 con 0x7fbc440061f0 2024-05-11T01:01:11.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fecd09ca640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40e31d0 0x7fecc40db200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fecc40e3660 con 0x7fecc40df210 2024-05-11T01:01:11.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fecc40065b0 con 0x7fecc40e31d0 2024-05-11T01:01:11.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fecc40df5f0 con 0x7fecc40e2d70 2024-05-11T01:01:11.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fce36ffd640 1 -- 172.21.15.52:0/1791624207 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30010c80 con 0x7fce400cdee0 2024-05-11T01:01:11.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fce34ff9640 1 -- 172.21.15.52:0/1791624207 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce4006f070 con 0x7fce400cdee0 2024-05-11T01:01:11.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fe41dffb640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe40c0763a0 0x7fe40c078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fdfa26bf640 1 --2- 172.21.15.52:0/4233785798 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cdf10 0x7fdf9c06e160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf980097a0 tx=0x7fdf9802a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e166186e66071322 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.848+0000 7fe41dffb640 1 -- 172.21.15.52:0/2031764674 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe424097230 con 0x7fe42000aa90 2024-05-11T01:01:11.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fe41dffb640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7fe40c07e5c0 0x7fe40c0809e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fe41dffb640 1 -- 172.21.15.52:0/2031764674 --> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe40c081090 con 0x7fe40c07e5c0 2024-05-11T01:01:11.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fe41dffb640 1 -- 172.21.15.52:0/2031764674 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe42409e050 con 0x7fe42000aa90 2024-05-11T01:01:11.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fe42cecd640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe40c0763a0 0x7fe40c078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fdfa0ebc640 1 -- 172.21.15.52:0/4233785798 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98037070 con 0x7fdf9c0cdf10 2024-05-11T01:01:11.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc440057d0 con 0x7fbc440061f0 2024-05-11T01:01:11.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fdfa0ebc640 1 -- 172.21.15.52:0/4233785798 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9802ae30 con 0x7fdf9c0cdf10 2024-05-11T01:01:11.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fdfa0ebc640 1 -- 172.21.15.52:0/4233785798 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98032640 con 0x7fdf9c0cdf10 2024-05-11T01:01:11.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7feccb7fe640 1 -- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e2d70 msgr2=0x7fecc4174d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7feccb7fe640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fecc40e2d70 0x7fecc4174d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7feccb7fe640 1 -- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40e31d0 msgr2=0x7fecc40db200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T01:01:11.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7feccb7fe640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40e31d0 0x7fecc40db200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fe42cecd640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe40c0763a0 0x7fe40c078860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe420008670 tx=0x7fe4280b6040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7feccb7fe640 1 -- 172.21.15.52:0/1350531231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc4175c40 con 0x7fecc40df210 2024-05-11T01:01:11.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff26015c620 con 0x7ff2600cb320 2024-05-11T01:01:11.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff260159130 con 0x7ff26015ece0 2024-05-11T01:01:11.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.849+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd88001030 con 0x7fcda40b4900 2024-05-11T01:01:11.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7ff2652cf640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 0x7ff2601f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fe42decf640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7fe40c07e5c0 0x7fe40c0809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fdfa4122640 1 -- 172.21.15.52:0/4233785798 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cdf10 msgr2=0x7fdf9c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fdfa4122640 1 --2- 172.21.15.52:0/4233785798 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cdf10 0x7fdf9c06e160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf980097a0 tx=0x7fdf9802a0e0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7feccb7fe640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecc40df210 0x7fecc4038700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c400 tx=0x7fecc000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7f4ca9de3640 1 Processor -- start 2024-05-11T01:01:11.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fdfa4122640 1 -- 172.21.15.52:0/4233785798 shutdown_connections 2024-05-11T01:01:11.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fecb7fff640 1 -- 172.21.15.52:0/1350531231 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0005ce0 con 0x7fecc40df210 2024-05-11T01:01:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7ff2652cf640 1 -- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2600cb320 msgr2=0x7ff2600cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fdfa4122640 1 --2- 172.21.15.52:0/4233785798 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cdf10 0x7fdf9c06e160 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fdfa4122640 1 --2- 172.21.15.52:0/4233785798 >> v2:172.21.15.119:3300/0 conn(0x7fdf9c15d190 0x7fdf9c06fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fdfa4122640 1 -- 172.21.15.52:0/4233785798 >> 172.21.15.52:0/4233785798 conn(0x7fdf9c0b2900 msgr2=0x7fdf9c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7ff2652cf640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2600cb320 0x7ff2600cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7ff2652cf640 1 -- 172.21.15.52:0/1277727154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff26015c170 msgr2=0x7ff2600afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7f4ca9de3640 1 -- start start 2024-05-11T01:01:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fce36ffd640 1 -- 172.21.15.52:0/1791624207 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fce30017070 con 0x7fce400cdee0 2024-05-11T01:01:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7ff2652cf640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff26015c170 0x7ff2600afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.850+0000 7fecb7fff640 1 -- 172.21.15.52:0/1350531231 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc0015030 con 0x7fecc40df210 2024-05-11T01:01:11.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7ff2652cf640 1 -- 172.21.15.52:0/1277727154 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2600b01c0 con 0x7ff26015ece0 2024-05-11T01:01:11.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7ff25ffff640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff26015c170 0x7ff2600afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:11.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7ff264ace640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2600cb320 0x7ff2600cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:11.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7fdfa4122640 1 -- 172.21.15.52:0/4233785798 shutdown_connections 2024-05-11T01:01:11.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7fdfa4122640 1 -- 172.21.15.52:0/4233785798 wait complete. 2024-05-11T01:01:11.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7ff2652cf640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 0x7ff2601f0940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff258033000 tx=0x7ff258004a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7fbc41ffb640 1 -- 172.21.15.52:0/2626069710 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbc4801a070 con 0x7fbc440061f0 2024-05-11T01:01:11.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7fce36ffd640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fce240763a0 0x7fce24078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7fcd9bfff640 1 -- 172.21.15.52:0/1657869674 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcd8c007450 con 0x7fcda40b4900 2024-05-11T01:01:11.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7ff25d7fa640 1 -- 172.21.15.52:0/1277727154 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258041070 con 0x7ff26015ece0 2024-05-11T01:01:11.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7fce36ffd640 1 -- 172.21.15.52:0/1791624207 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce30097880 con 0x7fce400cdee0 2024-05-11T01:01:11.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7ff25d7fa640 1 -- 172.21.15.52:0/1277727154 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff258033cc0 con 0x7ff26015ece0 2024-05-11T01:01:11.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.851+0000 7ff25d7fa640 1 -- 172.21.15.52:0/1277727154 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258007740 con 0x7ff26015ece0 2024-05-11T01:01:11.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fecb7fff640 1 -- 172.21.15.52:0/1350531231 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc00072d0 con 0x7fecc40df210 2024-05-11T01:01:11.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc4175680 con 0x7fecc40df210 2024-05-11T01:01:11.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fce36ffd640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7fce2407e620 0x7fce24080a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fce46966640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7fce2407e620 0x7fce24080a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2601f1190 con 0x7ff26015ece0 2024-05-11T01:01:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2601f1750 con 0x7ff26015ece0 2024-05-11T01:01:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fbc41ffb640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc300762d0 0x7fbc30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fbc41ffb640 1 -- 172.21.15.52:0/2626069710 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc48097c70 con 0x7fbc440061f0 2024-05-11T01:01:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fbc50b75640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc300762d0 0x7fbc30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fbc41ffb640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fbc3007e470 0x7fbc30080890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fbc41ffb640 1 -- 172.21.15.52:0/2626069710 --> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc30080f40 con 0x7fbc3007e470 2024-05-11T01:01:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fbc41ffb640 1 -- 172.21.15.52:0/2626069710 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbc48098020 con 0x7fbc440061f0 2024-05-11T01:01:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fcd9bfff640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcd840762d0 0x7fcd84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fcda2ffd640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcd840762d0 0x7fcd84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7fcd9bfff640 1 -- 172.21.15.52:0/1657869674 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcd8c097cd0 con 0x7fcda40b4900 2024-05-11T01:01:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fcda2ffd640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcd840762d0 0x7fcd84078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c006fd0 tx=0x7fcd9c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7f4ca9de3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ca406eb90 0x7f4ca415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7f4ca9de3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ca40cd4a0 0x7f4ca414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7f4ca8de1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ca406eb90 0x7f4ca415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7f4ca8de1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ca406eb90 0x7f4ca415a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:40998/0 (socket says 172.21.15.52:40998) 2024-05-11T01:01:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ca40cd4a0 0x7f4ca414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ca40cd4a0 0x7f4ca414d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:46402/0 (socket says 172.21.15.52:46402) 2024-05-11T01:01:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.852+0000 7f4ca9de3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca415c450 0x7f4ca40ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fce45964640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fce240763a0 0x7fce24078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fce36ffd640 1 -- 172.21.15.52:0/1791624207 --> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce3000ce50 con 0x7fce2407e620 2024-05-11T01:01:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fcd9bfff640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7fcd8407e470 0x7fcd84080890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fcd9bfff640 1 -- 172.21.15.52:0/1657869674 --> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd8c007740 con 0x7fcd8407e470 2024-05-11T01:01:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fcd9bfff640 1 -- 172.21.15.52:0/1657869674 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcd8c098080 con 0x7fcda40b4900 2024-05-11T01:01:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fce46966640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7fce2407e620 0x7fce24080a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7f4ca8de1640 1 -- 172.21.15.52:0/2424859250 learned_addr learned my addr 172.21.15.52:0/2424859250 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40cb770 con 0x7f4ca415c450 2024-05-11T01:01:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca415b4d0 con 0x7f4ca40cd4a0 2024-05-11T01:01:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7ff24affd640 1 -- 172.21.15.52:0/1277727154 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff238001030 con 0x7ff26015ece0 2024-05-11T01:01:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca40cb8f0 con 0x7f4ca406eb90 2024-05-11T01:01:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fcda37fe640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7fcd8407e470 0x7fcd84080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7f4ca95e2640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca415c450 0x7f4ca40ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fdfa4122640 1 Processor -- start 2024-05-11T01:01:11.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fdfa4122640 1 -- start start 2024-05-11T01:01:11.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.853+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc4035dc0 con 0x7fecc40df210 2024-05-11T01:01:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fbc50b75640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc300762d0 0x7fbc30078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbc44180a60 tx=0x7fbc3c007d00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fcda37fe640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7fcd8407e470 0x7fcd84080890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fbc51b77640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fbc3007e470 0x7fbc30080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fecc40dddc0 con 0x7fecc40df210 2024-05-11T01:01:11.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fce36ffd640 1 -- 172.21.15.52:0/1791624207 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fce30097c70 con 0x7fce400cdee0 2024-05-11T01:01:11.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fce45964640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fce240763a0 0x7fce24078860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce400c7300 tx=0x7fce3c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca8de1640 1 -- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ca40cd4a0 msgr2=0x7f4ca414d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca8de1640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ca40cd4a0 0x7f4ca414d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca8de1640 1 -- 172.21.15.52:0/2424859250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca415c450 msgr2=0x7f4ca40ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca8de1640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ca415c450 0x7f4ca40ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca8de1640 1 -- 172.21.15.52:0/2424859250 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca414d580 con 0x7f4ca406eb90 2024-05-11T01:01:11.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca8de1640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ca406eb90 0x7f4ca415a290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001e90 tx=0x7f4c9800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fe42decf640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7fe40c07e5c0 0x7fe40c0809e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fce36ffd640 1 -- 172.21.15.52:0/1791624207 <== osd.1 v2:172.21.15.52:6808/1366717920 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce3000ce50 con 0x7fce2407e620 2024-05-11T01:01:11.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca17fa640 1 -- 172.21.15.52:0/2424859250 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98005da0 con 0x7f4ca406eb90 2024-05-11T01:01:11.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca17fa640 1 -- 172.21.15.52:0/2424859250 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c98015030 con 0x7f4ca406eb90 2024-05-11T01:01:11.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca17fa640 1 -- 172.21.15.52:0/2424859250 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98007270 con 0x7f4ca406eb90 2024-05-11T01:01:11.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fdfa4122640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cce40 0x7fdf9c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7ff25d7fa640 1 -- 172.21.15.52:0/1277727154 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff258046680 con 0x7ff26015ece0 2024-05-11T01:01:11.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fdfa4122640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf9c0cdf10 0x7fdf9c15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa4122640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf9c15d190 0x7fdf9c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa1ebe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cce40 0x7fdf9c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa1ebe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cce40 0x7fdf9c15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41000/0 (socket says 172.21.15.52:41000) 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa1ebe640 1 -- 172.21.15.52:0/4204135803 learned_addr learned my addr 172.21.15.52:0/4204135803 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c161480 con 0x7fdf9c0cdf10 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1617f0 con 0x7fdf9c15d190 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c15d570 con 0x7fdf9c0cce40 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa26bf640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf9c15d190 0x7fdf9c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa16bd640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf9c0cdf10 0x7fdf9c15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7fbc51b77640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fbc3007e470 0x7fbc30080890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fcd9bfff640 1 -- 172.21.15.52:0/1657869674 <== osd.6 v2:172.21.15.119:6808/2126488559 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd8c007740 con 0x7fcd8407e470 2024-05-11T01:01:11.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.854+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca4157250 con 0x7f4ca406eb90 2024-05-11T01:01:11.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca40ca660 con 0x7f4ca406eb90 2024-05-11T01:01:11.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fecb7fff640 1 -- 172.21.15.52:0/1350531231 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fecc0007470 con 0x7fecc40df210 2024-05-11T01:01:11.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa26bf640 1 -- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cce40 msgr2=0x7fdf9c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fbc41ffb640 1 -- 172.21.15.52:0/2626069710 <== osd.5 v2:172.21.15.119:6824/3499512602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbc30080f40 con 0x7fbc3007e470 2024-05-11T01:01:11.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa26bf640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdf9c0cce40 0x7fdf9c15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa26bf640 1 -- 172.21.15.52:0/4204135803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf9c0cdf10 msgr2=0x7fdf9c15b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa26bf640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf9c0cdf10 0x7fdf9c15b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7fdfa26bf640 1 -- 172.21.15.52:0/4204135803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c15f750 con 0x7fdf9c15d190 2024-05-11T01:01:11.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.855+0000 7ff25d7fa640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2440762d0 0x7ff244078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7ff25d7fa640 1 -- 172.21.15.52:0/1277727154 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff2580bb1d0 con 0x7ff26015ece0 2024-05-11T01:01:11.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7ff25d7fa640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7ff24407e470 0x7ff244080890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7ff25d7fa640 1 -- 172.21.15.52:0/1277727154 --> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2580078e0 con 0x7ff24407e470 2024-05-11T01:01:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7ff25d7fa640 1 -- 172.21.15.52:0/1277727154 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff2580bb580 con 0x7ff26015ece0 2024-05-11T01:01:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7fecb7fff640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fecb00763d0 0x7fecb0078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7ff264ace640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2440762d0 0x7ff244078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7ff25ffff640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7ff24407e470 0x7ff244080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7fdfa26bf640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf9c15d190 0x7fdf9c15be70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf98031000 tx=0x7fdf980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7fecb7fff640 1 -- 172.21.15.52:0/1350531231 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fecc009abf0 con 0x7fecc40df210 2024-05-11T01:01:11.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.857+0000 7fecb7fff640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7fecb007e5f0 0x7fecb0080a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.856+0000 7fdf927fc640 1 -- 172.21.15.52:0/4204135803 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98041070 con 0x7fdf9c15d190 2024-05-11T01:01:11.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.857+0000 7fdf927fc640 1 -- 172.21.15.52:0/4204135803 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9803c660 con 0x7fdf9c15d190 2024-05-11T01:01:11.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.857+0000 7ff25ffff640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7ff24407e470 0x7ff244080890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.857+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c15fa30 con 0x7fdf9c15d190 2024-05-11T01:01:11.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.858+0000 7fecb7fff640 1 -- 172.21.15.52:0/1350531231 --> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fecc0015030 con 0x7fecb007e5f0 2024-05-11T01:01:11.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.858+0000 7fecb7fff640 1 -- 172.21.15.52:0/1350531231 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fecc009f2a0 con 0x7fecc40df210 2024-05-11T01:01:11.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.858+0000 7feccbfff640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7fecb007e5f0 0x7fecb0080a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.857+0000 7ff264ace640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2440762d0 0x7ff244078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff254009b80 tx=0x7ff254008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.858+0000 7ff25d7fa640 1 -- 172.21.15.52:0/1277727154 <== osd.2 v2:172.21.15.52:6824/410093751 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff2580078e0 con 0x7ff24407e470 2024-05-11T01:01:11.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.857+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c15fff0 con 0x7fdf9c15d190 2024-05-11T01:01:11.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.858+0000 7fdf927fc640 1 -- 172.21.15.52:0/4204135803 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98037040 con 0x7fdf9c15d190 2024-05-11T01:01:11.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.858+0000 7fdf87fff640 1 -- 172.21.15.52:0/4204135803 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdf9c069670 con 0x7fdf9c15d190 2024-05-11T01:01:11.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.857+0000 7fe41dffb640 1 -- 172.21.15.52:0/2031764674 <== osd.4 v2:172.21.15.119:6800/2665075860 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe40c081090 con 0x7fe40c07e5c0 2024-05-11T01:01:11.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.857+0000 7f4ca17fa640 1 -- 172.21.15.52:0/2424859250 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4c98007410 con 0x7f4ca406eb90 2024-05-11T01:01:11.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.858+0000 7f4ca17fa640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4c880763a0 0x7f4c88078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.859+0000 7feccbfff640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7fecb007e5f0 0x7fecb0080a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.857+0000 7feccaffd640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fecb00763d0 0x7fecb0078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.859+0000 7fecb7fff640 1 -- 172.21.15.52:0/1350531231 <== osd.0 v2:172.21.15.52:6800/1335322190 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fecc0015030 con 0x7fecb007e5f0 2024-05-11T01:01:11.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.858+0000 7f4ca17fa640 1 -- 172.21.15.52:0/2424859250 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4c9809ad80 con 0x7f4ca406eb90 2024-05-11T01:01:11.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.858+0000 7f4c8effd640 1 -- 172.21.15.52:0/2424859250 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ca40694f0 con 0x7f4ca406eb90 2024-05-11T01:01:11.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.859+0000 7f4ca17fa640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f4c8807e540 0x7f4c88080960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.859+0000 7f4ca17fa640 1 -- 172.21.15.52:0/2424859250 --> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c88081010 con 0x7f4c8807e540 2024-05-11T01:01:11.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.859+0000 7f4ca17fa640 1 -- 172.21.15.52:0/2424859250 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4c9801f020 con 0x7f4ca406eb90 2024-05-11T01:01:11.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.859+0000 7f4ca3fff640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4c880763a0 0x7f4c88078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.860+0000 7f4ca95e2640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f4c8807e540 0x7f4c88080960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.860+0000 7f4ca3fff640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4c880763a0 0x7f4c88078860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c90034000 tx=0x7f4c9002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.861+0000 7fdf927fc640 1 -- 172.21.15.52:0/4204135803 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdf98044070 con 0x7fdf9c15d190 2024-05-11T01:01:11.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.861+0000 7feccaffd640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fecb00763d0 0x7fecb0078890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fecb8006fd0 tx=0x7fecb8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.861+0000 7fdf927fc640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdf800763a0 0x7fdf80078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.862+0000 7fdf927fc640 1 -- 172.21.15.52:0/4204135803 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf980b7ec0 con 0x7fdf9c15d190 2024-05-11T01:01:11.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.862+0000 7fdfa1ebe640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdf800763a0 0x7fdf80078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.862+0000 7fdf927fc640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7fdf8007e540 0x7fdf80080960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:11.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.862+0000 7f4ca95e2640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f4c8807e540 0x7f4c88080960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.862+0000 7fdf927fc640 1 -- 172.21.15.52:0/4204135803 --> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf80081010 con 0x7fdf8007e540 2024-05-11T01:01:11.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.862+0000 7fdf927fc640 1 -- 172.21.15.52:0/4204135803 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdf9807d800 con 0x7fdf9c15d190 2024-05-11T01:01:11.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.862+0000 7fdfa16bd640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7fdf8007e540 0x7fdf80080960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:11.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.863+0000 7fdfa1ebe640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdf800763a0 0x7fdf80078860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf940070b0 tx=0x7fdf9400d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.863+0000 7fdfa16bd640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7fdf8007e540 0x7fdf80080960 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:11.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.863+0000 7f4ca17fa640 1 -- 172.21.15.52:0/2424859250 <== osd.7 v2:172.21.15.119:6816/4214692661 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c88081010 con 0x7f4c8807e540 2024-05-11T01:01:11.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.864+0000 7fdf927fc640 1 -- 172.21.15.52:0/4204135803 <== osd.3 v2:172.21.15.52:6816/2352208085 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdf80081010 con 0x7fdf8007e540 2024-05-11T01:01:11.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.876+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 --> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fecc40057d0 con 0x7fecb007e5f0 2024-05-11T01:01:11.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.876+0000 7fecb7fff640 1 -- 172.21.15.52:0/1350531231 <== osd.0 v2:172.21.15.52:6800/1335322190 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fecc40057d0 con 0x7fecb007e5f0 2024-05-11T01:01:11.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.877+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7fecb007e5f0 msgr2=0x7fecb0080a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.877+0000 7fecd09ca640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7fecb007e5f0 0x7fecb0080a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.878+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fecb00763d0 msgr2=0x7fecb0078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.878+0000 7fecd09ca640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fecb00763d0 0x7fecb0078890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fecb8006fd0 tx=0x7fecb8008040 comp rx=0 tx=0).stop 2024-05-11T01:01:11.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.878+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecc40df210 msgr2=0x7fecc4038700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.878+0000 7fecd09ca640 1 --2- 172.21.15.52:0/1350531231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecc40df210 0x7fecc4038700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c400 tx=0x7fecc000c8c0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.878+0000 7feccbfff640 1 -- 172.21.15.52:0/1350531231 reap_dead start 2024-05-11T01:01:11.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.878+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 shutdown_connections 2024-05-11T01:01:11.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.878+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 >> 172.21.15.52:0/1350531231 conn(0x7fecc400a3f0 msgr2=0x7fecc4006ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.881+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 shutdown_connections 2024-05-11T01:01:11.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.881+0000 7fecd09ca640 1 -- 172.21.15.52:0/1350531231 wait complete. 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.885+0000 7fce34ff9640 1 -- 172.21.15.52:0/1791624207 --> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce40069670 con 0x7fce2407e620 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.886+0000 7fce36ffd640 1 -- 172.21.15.52:0/1791624207 <== osd.1 v2:172.21.15.52:6808/1366717920 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fce40069670 con 0x7fce2407e620 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.886+0000 7fce483c9640 1 -- 172.21.15.52:0/1791624207 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7fce2407e620 msgr2=0x7fce24080a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.886+0000 7fce483c9640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7fce2407e620 0x7fce24080a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.887+0000 7fce483c9640 1 -- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fce240763a0 msgr2=0x7fce24078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.887+0000 7fce483c9640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fce240763a0 0x7fce24078860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce400c7300 tx=0x7fce3c009040 comp rx=0 tx=0).stop 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.887+0000 7fce483c9640 1 -- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fce400cdee0 msgr2=0x7fce400c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.887+0000 7fce483c9640 1 --2- 172.21.15.52:0/1791624207 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fce400cdee0 0x7fce400c5b10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fce3000b480 tx=0x7fce3000b940 comp rx=0 tx=0).stop 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.887+0000 7fce46966640 1 -- 172.21.15.52:0/1791624207 reap_dead start 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.887+0000 7fce483c9640 1 -- 172.21.15.52:0/1791624207 shutdown_connections 2024-05-11T01:01:11.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.887+0000 7fce483c9640 1 -- 172.21.15.52:0/1791624207 >> 172.21.15.52:0/1791624207 conn(0x7fce400b2900 msgr2=0x7fce400cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.887+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 --> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd88002dd0 con 0x7fcd8407e470 2024-05-11T01:01:11.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.887+0000 7fce483c9640 1 -- 172.21.15.52:0/1791624207 shutdown_connections 2024-05-11T01:01:11.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.888+0000 7fce483c9640 1 -- 172.21.15.52:0/1791624207 wait complete. 2024-05-11T01:01:11.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.888+0000 7fcd9bfff640 1 -- 172.21.15.52:0/1657869674 <== osd.6 v2:172.21.15.119:6808/2126488559 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcd88002dd0 con 0x7fcd8407e470 2024-05-11T01:01:11.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.888+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7fcd8407e470 msgr2=0x7fcd84080890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.889+0000 7fcda98d1640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7fcd8407e470 0x7fcd84080890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.889+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcd840762d0 msgr2=0x7fcd84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.889+0000 7fcda98d1640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fcd840762d0 0x7fcd84078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c006fd0 tx=0x7fcd9c008040 comp rx=0 tx=0).stop 2024-05-11T01:01:11.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.889+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcda40b4900 msgr2=0x7fcda42012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.889+0000 7fcda98d1640 1 --2- 172.21.15.52:0/1657869674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcda40b4900 0x7fcda42012e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c700 tx=0x7fcd8c00cb00 comp rx=0 tx=0).stop 2024-05-11T01:01:11.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.889+0000 7fcda37fe640 1 -- 172.21.15.52:0/1657869674 reap_dead start 2024-05-11T01:01:11.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.889+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 shutdown_connections 2024-05-11T01:01:11.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.889+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 >> 172.21.15.52:0/1657869674 conn(0x7fcda40b2d80 msgr2=0x7fcda40b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.890+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 --> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe420053ff0 con 0x7fe40c07e5c0 2024-05-11T01:01:11.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.890+0000 7fe41dffb640 1 -- 172.21.15.52:0/2031764674 <== osd.4 v2:172.21.15.119:6800/2665075860 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe420053ff0 con 0x7fe40c07e5c0 2024-05-11T01:01:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.891+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 shutdown_connections 2024-05-11T01:01:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.891+0000 7fcda98d1640 1 -- 172.21.15.52:0/1657869674 wait complete. 2024-05-11T01:01:11.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.891+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7fe40c07e5c0 msgr2=0x7fe40c0809e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.891+0000 7fe42e6d0640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7fe40c07e5c0 0x7fe40c0809e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.891+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe40c0763a0 msgr2=0x7fe40c078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.891+0000 7fe42e6d0640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe40c0763a0 0x7fe40c078860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe420008670 tx=0x7fe4280b6040 comp rx=0 tx=0).stop 2024-05-11T01:01:11.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.892+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe42000aa90 msgr2=0x7fe42004d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.892+0000 7fe42e6d0640 1 --2- 172.21.15.52:0/2031764674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe42000aa90 0x7fe42004d5e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c500 tx=0x7fe42400c9c0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.892+0000 7fe42decf640 1 -- 172.21.15.52:0/2031764674 reap_dead start 2024-05-11T01:01:11.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.893+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 shutdown_connections 2024-05-11T01:01:11.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.893+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 >> 172.21.15.52:0/2031764674 conn(0x7fe420009fa0 msgr2=0x7fe4200dc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.893+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 shutdown_connections 2024-05-11T01:01:11.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.893+0000 7fe42e6d0640 1 -- 172.21.15.52:0/2031764674 wait complete. 2024-05-11T01:01:11.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.894+0000 7ff24affd640 1 -- 172.21.15.52:0/1277727154 --> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff238002dd0 con 0x7ff24407e470 2024-05-11T01:01:11.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.895+0000 7ff25d7fa640 1 -- 172.21.15.52:0/1277727154 <== osd.2 v2:172.21.15.52:6824/410093751 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff238002dd0 con 0x7ff24407e470 2024-05-11T01:01:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.896+0000 7fdf87fff640 1 -- 172.21.15.52:0/4204135803 --> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdf9c06f6f0 con 0x7fdf8007e540 2024-05-11T01:01:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.896+0000 7f4c8effd640 1 -- 172.21.15.52:0/2424859250 --> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ca40c4360 con 0x7f4c8807e540 2024-05-11T01:01:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.896+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7ff24407e470 msgr2=0x7ff244080890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.896+0000 7ff266d32640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7ff24407e470 0x7ff244080890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.896+0000 7fdf927fc640 1 -- 172.21.15.52:0/4204135803 <== osd.3 v2:172.21.15.52:6816/2352208085 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdf9c06f6f0 con 0x7fdf8007e540 2024-05-11T01:01:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.896+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2440762d0 msgr2=0x7ff244078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.896+0000 7ff266d32640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff2440762d0 0x7ff244078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff254009b80 tx=0x7ff254008040 comp rx=0 tx=0).stop 2024-05-11T01:01:11.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 --> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbc440e5910 con 0x7fbc3007e470 2024-05-11T01:01:11.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.896+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 msgr2=0x7ff2601f0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.896+0000 7ff266d32640 1 --2- 172.21.15.52:0/1277727154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff26015ece0 0x7ff2601f0940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff258033000 tx=0x7ff258004a70 comp rx=0 tx=0).stop 2024-05-11T01:01:11.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7f4ca17fa640 1 -- 172.21.15.52:0/2424859250 <== osd.7 v2:172.21.15.119:6816/4214692661 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ca40c4360 con 0x7f4c8807e540 2024-05-11T01:01:11.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7ff2652cf640 1 -- 172.21.15.52:0/1277727154 reap_dead start 2024-05-11T01:01:11.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 shutdown_connections 2024-05-11T01:01:11.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 >> 172.21.15.52:0/1277727154 conn(0x7ff2600cd750 msgr2=0x7ff26015f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7fbc41ffb640 1 -- 172.21.15.52:0/2626069710 <== osd.5 v2:172.21.15.119:6824/3499512602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbc440e5910 con 0x7fbc3007e470 2024-05-11T01:01:11.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7fdf8007e540 msgr2=0x7fdf80080960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7fdfa4122640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7fdf8007e540 0x7fdf80080960 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f4c8807e540 msgr2=0x7f4c88080960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7f4ca9de3640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f4c8807e540 0x7f4c88080960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdf800763a0 msgr2=0x7fdf80078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fdfa4122640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdf800763a0 0x7fdf80078860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf940070b0 tx=0x7fdf9400d040 comp rx=0 tx=0).stop 2024-05-11T01:01:11.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.897+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 shutdown_connections 2024-05-11T01:01:11.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fbc3007e470 msgr2=0x7fbc30080890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fbc3007e470 0x7fbc30080890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7ff266d32640 1 -- 172.21.15.52:0/1277727154 wait complete. 2024-05-11T01:01:11.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf9c15d190 msgr2=0x7fdf9c15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fdfa4122640 1 --2- 172.21.15.52:0/4204135803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf9c15d190 0x7fdf9c15be70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf98031000 tx=0x7fdf980046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:11.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fdfa26bf640 1 -- 172.21.15.52:0/4204135803 reap_dead start 2024-05-11T01:01:11.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4c880763a0 msgr2=0x7f4c88078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7f4ca9de3640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4c880763a0 0x7f4c88078860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c90034000 tx=0x7f4c9002e000 comp rx=0 tx=0).stop 2024-05-11T01:01:11.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ca406eb90 msgr2=0x7f4ca415a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7f4ca9de3640 1 --2- 172.21.15.52:0/2424859250 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ca406eb90 0x7f4ca415a290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001e90 tx=0x7f4c9800c990 comp rx=0 tx=0).stop 2024-05-11T01:01:11.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7f4ca95e2640 1 -- 172.21.15.52:0/2424859250 reap_dead start 2024-05-11T01:01:11.963 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:11.964 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T01:01:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc300762d0 msgr2=0x7fbc30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc300762d0 0x7fbc30078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbc44180a60 tx=0x7fbc3c007d00 comp rx=0 tx=0).stop 2024-05-11T01:01:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 shutdown_connections 2024-05-11T01:01:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 >> 172.21.15.52:0/2424859250 conn(0x7f4ca40b26f0 msgr2=0x7f4ca40d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc440061f0 msgr2=0x7fbc441816a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 --2- 172.21.15.52:0/2626069710 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc440061f0 0x7fbc441816a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800e850 tx=0x7fbc4800ed10 comp rx=0 tx=0).stop 2024-05-11T01:01:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc51b77640 1 -- 172.21.15.52:0/2626069710 reap_dead start 2024-05-11T01:01:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 shutdown_connections 2024-05-11T01:01:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 >> 172.21.15.52:0/4204135803 conn(0x7fdf9c0b2900 msgr2=0x7fdf9c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 shutdown_connections 2024-05-11T01:01:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 >> 172.21.15.52:0/2626069710 conn(0x7fbc44058ed0 msgr2=0x7fbc44053750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 shutdown_connections 2024-05-11T01:01:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7f4ca9de3640 1 -- 172.21.15.52:0/2424859250 wait complete. 2024-05-11T01:01:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 shutdown_connections 2024-05-11T01:01:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fdfa4122640 1 -- 172.21.15.52:0/4204135803 wait complete. 2024-05-11T01:01:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 shutdown_connections 2024-05-11T01:01:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:11.898+0000 7fbc535da640 1 -- 172.21.15.52:0/2626069710 wait complete. 2024-05-11T01:01:11.968 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:11.968 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T01:01:11.969 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:11.969 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T01:01:11.970 INFO:teuthology.orchestra.run.smithi052.stdout:60129542152 2024-05-11T01:01:11.970 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T01:01:11.971 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:11.971 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T01:01:11.972 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:11.972 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T01:01:11.973 INFO:teuthology.orchestra.run.smithi052.stdout:55834574855 2024-05-11T01:01:11.973 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T01:01:11.974 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:11.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T01:01:12.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.187+0000 7fc7b892b640 1 Processor -- start 2024-05-11T01:01:12.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.187+0000 7fc7b892b640 1 -- start start 2024-05-11T01:01:12.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.187+0000 7f537d872640 1 Processor -- start 2024-05-11T01:01:12.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.187+0000 7f537d872640 1 -- start start 2024-05-11T01:01:12.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.188+0000 7f55c9d58640 1 Processor -- start 2024-05-11T01:01:12.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.188+0000 7f55c9d58640 1 -- start start 2024-05-11T01:01:12.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7fb89557b640 1 Processor -- start 2024-05-11T01:01:12.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7fb89557b640 1 -- start start 2024-05-11T01:01:12.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7fc7b892b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc7ac0df210 0x7fc7ac038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7fc7b892b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac050190 0x7fc7ac006a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7fc7b892b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7ac0544b0 con 0x7fc7ac053c60 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7fc7b2d9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac050190 0x7fc7ac006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7fc7b2d9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac050190 0x7fc7ac006a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41032/0 (socket says 172.21.15.52:41032) 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7fc7b2d9d640 1 -- 172.21.15.52:0/1495491018 learned_addr learned my addr 172.21.15.52:0/1495491018 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.188+0000 7f537d872640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53780b1260 0x7f53780c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.188+0000 7f537d872640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5378160730 0x7f537806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.188+0000 7f537d872640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f53780702a0 0x7f53780d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f537d872640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f53780d08f0 con 0x7f53780702a0 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f537d872640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f53780d02e0 con 0x7f5378160730 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f537d872640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f53780c7d50 con 0x7f53780b1260 2024-05-11T01:01:12.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f53777fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f53780702a0 0x7f53780d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f53777fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f53780702a0 0x7f53780d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35126/0 (socket says 172.21.15.52:35126) 2024-05-11T01:01:12.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f53777fe640 1 -- 172.21.15.52:0/1216148775 learned_addr learned my addr 172.21.15.52:0/1216148775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f5376ffd640 1 --2- 172.21.15.52:0/1216148775 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53780b1260 0x7f53780c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f53767fc640 1 --2- 172.21.15.52:0/1216148775 >> v2:172.21.15.119:3300/0 conn(0x7f5378160730 0x7f537806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f3c02f08640 1 Processor -- start 2024-05-11T01:01:12.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f42819dd640 1 Processor -- start 2024-05-11T01:01:12.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f42819dd640 1 -- start start 2024-05-11T01:01:12.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f55c9d58640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f55bc0e12a0 0x7f55bc0534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f55c9d58640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc054b40 0x7f55bc053a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f55c9d58640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55bc054fd0 con 0x7f55bc0e4d20 2024-05-11T01:01:12.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f55c9d58640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f55bc0552b0 con 0x7f55bc0e12a0 2024-05-11T01:01:12.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f55c9d58640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc055590 con 0x7f55bc054b40 2024-05-11T01:01:12.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f55c37fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f55bc0e12a0 0x7f55bc0534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f55c3fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f55bc0e4d20 0x7f55bc0e1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58074/0 (socket says 172.21.15.52:58074) 2024-05-11T01:01:12.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f55c37fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f55bc0e12a0 0x7f55bc0534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35140/0 (socket says 172.21.15.52:35140) 2024-05-11T01:01:12.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.189+0000 7f55c3fff640 1 -- 172.21.15.52:0/315127253 learned_addr learned my addr 172.21.15.52:0/315127253 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:12.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f55c2ffd640 1 --2- 172.21.15.52:0/315127253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc054b40 0x7f55bc053a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.188+0000 7fc16f342640 1 Processor -- start 2024-05-11T01:01:12.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f55c2ffd640 1 -- 172.21.15.52:0/315127253 >> v2:172.21.15.52:3300/0 conn(0x7f55bc0e12a0 msgr2=0x7f55bc0534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f55c2ffd640 1 --2- 172.21.15.52:0/315127253 >> v2:172.21.15.52:3300/0 conn(0x7f55bc0e12a0 0x7f55bc0534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f55c2ffd640 1 -- 172.21.15.52:0/315127253 >> v1:172.21.15.52:6789/0 conn(0x7f55bc0e4d20 legacy=0x7f55bc0e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f55c2ffd640 1 -- 172.21.15.52:0/315127253 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc055870 con 0x7f55bc054b40 2024-05-11T01:01:12.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.188+0000 7fc16f342640 1 -- start start 2024-05-11T01:01:12.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7fc16f342640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc1600078d0 0x7fc1600edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16db3f640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc16003e830 0x7fc160053be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:52904/0 (socket says 172.21.15.52:52904) 2024-05-11T01:01:12.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16e340640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc1600e30b0 0x7fc160057d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58084/0 (socket says 172.21.15.52:58084) 2024-05-11T01:01:12.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16db3f640 1 -- 172.21.15.52:0/2414099383 learned_addr learned my addr 172.21.15.52:0/2414099383 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:12.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16f342640 1 -- 172.21.15.52:0/2414099383 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc160058560 con 0x7fc1600e30b0 2024-05-11T01:01:12.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fb89557b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880e31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fb88ffff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb8880df210 0x7fb88803a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58088/0 (socket says 172.21.15.52:58088) 2024-05-11T01:01:12.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16eb41640 1 --2- 172.21.15.52:0/2414099383 >> v2:172.21.15.52:3300/0 conn(0x7fc1600078d0 0x7fc1600edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fb88ffff640 1 -- 172.21.15.52:0/3410336006 learned_addr learned my addr 172.21.15.52:0/3410336006 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:12.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fb88f7fe640 1 --2- 172.21.15.52:0/3410336006 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880e31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16f342640 1 -- 172.21.15.52:0/2414099383 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc160058840 con 0x7fc16003e830 2024-05-11T01:01:12.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16f342640 1 -- 172.21.15.52:0/2414099383 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc160058b20 con 0x7fc1600078d0 2024-05-11T01:01:12.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7f42819dd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f427c15b4c0 0x7f427c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7f42819dd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c072f00 0x7f427c06e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7f42819dd640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f427c06f5f0 con 0x7f427c06ee20 2024-05-11T01:01:12.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7f42819dd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f427c073340 con 0x7f427c15b4c0 2024-05-11T01:01:12.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7f42819dd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f427c06f8d0 con 0x7f427c072f00 2024-05-11T01:01:12.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f5376ffd640 1 -- 172.21.15.52:0/1216148775 >> v2:172.21.15.119:3300/0 conn(0x7f5378160730 msgr2=0x7f537806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f5376ffd640 1 --2- 172.21.15.52:0/1216148775 >> v2:172.21.15.119:3300/0 conn(0x7f5378160730 0x7f537806fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f5376ffd640 1 -- 172.21.15.52:0/1216148775 >> v2:172.21.15.52:3300/0 conn(0x7f53780702a0 msgr2=0x7f53780d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f5376ffd640 1 --2- 172.21.15.52:0/1216148775 >> v2:172.21.15.52:3300/0 conn(0x7f53780702a0 0x7f53780d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f5376ffd640 1 -- 172.21.15.52:0/1216148775 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53780c8030 con 0x7f53780b1260 2024-05-11T01:01:12.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16d33e640 1 -- 172.21.15.52:0/2414099383 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851935497 0 0) 0x7fc160058560 con 0x7fc1600e30b0 2024-05-11T01:01:12.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fb89557b640 1 --2- 172.21.15.52:0/3410336006 >> v2:172.21.15.119:3300/0 conn(0x7fb888006ad0 0x7fb888006f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16d33e640 1 -- 172.21.15.52:0/2414099383 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc154003520 con 0x7fc1600e30b0 2024-05-11T01:01:12.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.190+0000 7f3c02f08640 1 -- start start 2024-05-11T01:01:12.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7f3c02f08640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3bfc1636f0 0x7f3bfc0d24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16d33e640 1 -- 172.21.15.52:0/2414099383 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 526846213 0 0) 0x7fc160058840 con 0x7fc16003e830 2024-05-11T01:01:12.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc16d33e640 1 -- 172.21.15.52:0/2414099383 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc160058560 con 0x7fc16003e830 2024-05-11T01:01:12.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fb89557b640 1 -- 172.21.15.52:0/3410336006 --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb888054630 con 0x7fb8880df210 2024-05-11T01:01:12.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f3c02f08640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bfc0b1de0 con 0x7f3bfc0b7230 2024-05-11T01:01:12.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f427affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f427c15b4c0 0x7f427c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f55c2ffd640 1 --2- 172.21.15.52:0/315127253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc054b40 0x7f55bc053a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55b00097a0 tx=0x7f55b002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64dfb658a2fbaecc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fb89557b640 1 -- 172.21.15.52:0/3410336006 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb8880e3710 con 0x7fb888006ad0 2024-05-11T01:01:12.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f5376ffd640 1 --2- 172.21.15.52:0/1216148775 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53780b1260 0x7f53780c7690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53780743e0 tx=0x7f536802ea10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78dc4985294e554f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f427b7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f427c06ee20 0x7f427c15b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58096/0 (socket says 172.21.15.52:58096) 2024-05-11T01:01:12.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f427b7fe640 1 -- 172.21.15.52:0/291129204 learned_addr learned my addr 172.21.15.52:0/291129204 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:12.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fb89557b640 1 -- 172.21.15.52:0/3410336006 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb8880543e0 con 0x7fb8880e2d70 2024-05-11T01:01:12.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.191+0000 7fc7b259c640 1 --2- 172.21.15.52:0/1495491018 >> v2:172.21.15.52:3300/0 conn(0x7fc7ac0df210 0x7fc7ac038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f3c01f06640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3bfc1636f0 0x7f3bfc0d24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fc7b892b640 1 -- 172.21.15.52:0/1495491018 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc7ac0541d0 con 0x7fc7ac0df210 2024-05-11T01:01:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fb894d7a640 1 --2- 172.21.15.52:0/3410336006 >> v2:172.21.15.119:3300/0 conn(0x7fb888006ad0 0x7fb888006f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fc7b892b640 1 -- 172.21.15.52:0/1495491018 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac005f20 con 0x7fc7ac050190 2024-05-11T01:01:12.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fb88effd640 1 -- 172.21.15.52:0/3410336006 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896513891 0 0) 0x7fb888054630 con 0x7fb8880df210 2024-05-11T01:01:12.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f3c02707640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f3bfc0b7230 0x7f3bfc09cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58112/0 (socket says 172.21.15.52:58112) 2024-05-11T01:01:12.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fb88effd640 1 -- 172.21.15.52:0/3410336006 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb880003520 con 0x7fb8880df210 2024-05-11T01:01:12.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fc16eb41640 1 -- 172.21.15.52:0/2414099383 >> v1:172.21.15.119:6789/0 conn(0x7fc16003e830 legacy=0x7fc160053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fc16eb41640 1 -- 172.21.15.52:0/2414099383 >> v1:172.21.15.52:6789/0 conn(0x7fc1600e30b0 legacy=0x7fc160057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fc16eb41640 1 -- 172.21.15.52:0/2414099383 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1600063a0 con 0x7fc1600078d0 2024-05-11T01:01:12.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f427a7fc640 1 --2- 172.21.15.52:0/291129204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c072f00 0x7f427c06e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f3c02707640 1 -- 172.21.15.52:0/1973281202 learned_addr learned my addr 172.21.15.52:0/1973281202 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:12.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458202540 0 0) 0x7fc7ac0544b0 con 0x7fc7ac053c60 2024-05-11T01:01:12.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc79c003520 con 0x7fc7ac053c60 2024-05-11T01:01:12.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f5375ffb640 1 -- 172.21.15.52:0/1216148775 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368037070 con 0x7f53780b1260 2024-05-11T01:01:12.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f5375ffb640 1 -- 172.21.15.52:0/1216148775 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5368033870 con 0x7f53780b1260 2024-05-11T01:01:12.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f5375ffb640 1 -- 172.21.15.52:0/1216148775 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368033c80 con 0x7f53780b1260 2024-05-11T01:01:12.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268712638 0 0) 0x7f3bfc0b1de0 con 0x7f3bfc0b7230 2024-05-11T01:01:12.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f3c02f08640 1 -- 172.21.15.52:0/1973281202 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bfc0b20c0 con 0x7f3bfc163ad0 2024-05-11T01:01:12.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7ff62699e640 1 Processor -- start 2024-05-11T01:01:12.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f3c02f08640 1 -- 172.21.15.52:0/1973281202 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3bfc0b23a0 con 0x7f3bfc1636f0 2024-05-11T01:01:12.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f55c27fc640 1 -- 172.21.15.52:0/315127253 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0037070 con 0x7f55bc054b40 2024-05-11T01:01:12.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f55c27fc640 1 -- 172.21.15.52:0/315127253 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b0032440 con 0x7f55bc054b40 2024-05-11T01:01:12.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f55c27fc640 1 -- 172.21.15.52:0/315127253 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0037070 con 0x7f55bc054b40 2024-05-11T01:01:12.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fb88f7fe640 1 -- 172.21.15.52:0/3410336006 >> v2:172.21.15.119:3300/0 conn(0x7fb888006ad0 msgr2=0x7fb888006f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fb88f7fe640 1 --2- 172.21.15.52:0/3410336006 >> v2:172.21.15.119:3300/0 conn(0x7fb888006ad0 0x7fb888006f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7fb88f7fe640 1 -- 172.21.15.52:0/3410336006 >> v1:172.21.15.52:6789/0 conn(0x7fb8880df210 legacy=0x7fb88803a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fb88f7fe640 1 -- 172.21.15.52:0/3410336006 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb888054910 con 0x7fb8880e2d70 2024-05-11T01:01:12.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4044501983 0 0) 0x7fc79c003520 con 0x7fc7ac053c60 2024-05-11T01:01:12.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac050190 msgr2=0x7fc7ac006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac050190 0x7fc7ac006a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 >> v2:172.21.15.52:3300/0 conn(0x7fc7ac0df210 msgr2=0x7fc7ac038070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 --2- 172.21.15.52:0/1495491018 >> v2:172.21.15.52:3300/0 conn(0x7fc7ac0df210 0x7fc7ac038070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac006200 con 0x7fc7ac053c60 2024-05-11T01:01:12.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fc7a80016d0 con 0x7fc7ac053c60 2024-05-11T01:01:12.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 >> v1:172.21.15.119:6789/0 conn(0x7fc7ac053c60 legacy=0x7fc7ac04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3be8003520 con 0x7f3bfc0b7230 2024-05-11T01:01:12.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc79c004bd0 0x7fc79c0037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 0x7fc79c004240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc79c005370 0x7fc79c0057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac0541d0 con 0x7fc79c003dc0 2024-05-11T01:01:12.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac005f20 con 0x7fc79c004bd0 2024-05-11T01:01:12.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc79c0060d0 con 0x7fc79c005370 2024-05-11T01:01:12.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7fc7b259c640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc79c004bd0 0x7fc79c0037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292817297 0 0) 0x7f3bfc0b20c0 con 0x7f3bfc163ad0 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bfc0b1de0 con 0x7f3bfc163ad0 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4092456575 0 0) 0x7f3be8003520 con 0x7f3bfc0b7230 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 >> v2:172.21.15.119:3300/0 conn(0x7f3bfc1636f0 msgr2=0x7f3bfc0d24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 --2- 172.21.15.52:0/1973281202 >> v2:172.21.15.119:3300/0 conn(0x7f3bfc1636f0 0x7f3bfc0d24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 >> v1:172.21.15.119:6789/0 conn(0x7f3bfc163ad0 legacy=0x7f3bfc0b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc0b2680 con 0x7f3bfc0b7230 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f3bf4001a70 con 0x7f3bfc0b7230 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 >> v1:172.21.15.52:6789/0 conn(0x7f3bfc0b7230 legacy=0x7f3bfc09cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3be8004c90 0x7f3be8005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 0x7f3be8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be8004280 0x7f3be800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc0b23a0 con 0x7f3be80037a0 2024-05-11T01:01:12.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3be800c230 con 0x7f3be8004280 2024-05-11T01:01:12.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3be800c430 con 0x7f3be8004c90 2024-05-11T01:01:12.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f55c9d58640 1 -- 172.21.15.52:0/315127253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc054b40 msgr2=0x7f55bc053a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f55c9d58640 1 --2- 172.21.15.52:0/315127253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc054b40 0x7f55bc053a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55b00097a0 tx=0x7f55b002a180 comp rx=0 tx=0).stop 2024-05-11T01:01:12.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f427affd640 1 -- 172.21.15.52:0/291129204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c072f00 msgr2=0x7f427c06e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f427affd640 1 --2- 172.21.15.52:0/291129204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c072f00 0x7f427c06e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f427affd640 1 -- 172.21.15.52:0/291129204 >> v1:172.21.15.52:6789/0 conn(0x7f427c06ee20 legacy=0x7f427c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7f427affd640 1 -- 172.21.15.52:0/291129204 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c06fbb0 con 0x7f427c15b4c0 2024-05-11T01:01:12.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.193+0000 7f3c01705640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 0x7f3be8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc7b1d9b640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 0x7fc79c004240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f55c9d58640 1 -- 172.21.15.52:0/315127253 shutdown_connections 2024-05-11T01:01:12.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f55c9d58640 1 --2- 172.21.15.52:0/315127253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc054b40 0x7f55bc053a20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f55c9d58640 1 --2- 172.21.15.52:0/315127253 >> v2:172.21.15.52:3300/0 conn(0x7f55bc0e12a0 0x7f55bc0534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f55c9d58640 1 -- 172.21.15.52:0/315127253 >> 172.21.15.52:0/315127253 conn(0x7f55bc020030 msgr2=0x7f55bc056d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f3c01f06640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3be8004c90 0x7f3be8005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f427affd640 1 --2- 172.21.15.52:0/291129204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f427c15b4c0 0x7f427c0729c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f427000a940 tx=0x7f427002c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c49332402568fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fb88f7fe640 1 --2- 172.21.15.52:0/3410336006 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880e31d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00b160 tx=0x7fb87c02aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea814a15f281010 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f3c02707640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be8004280 0x7f3be800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f55c9d58640 1 -- 172.21.15.52:0/315127253 shutdown_connections 2024-05-11T01:01:12.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f55c9d58640 1 -- 172.21.15.52:0/315127253 wait complete. 2024-05-11T01:01:12.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc16eb41640 1 --2- 172.21.15.52:0/2414099383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 0x7fc1600edd80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc16802e140 tx=0x7fc1680a6740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a70df2ac7075b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.192+0000 7ff62699e640 1 -- start start 2024-05-11T01:01:12.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f3c01705640 1 -- 172.21.15.52:0/1973281202 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3be8004c90 msgr2=0x7f3be8005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f3c01705640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3be8004c90 0x7f3be8005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f3c01705640 1 -- 172.21.15.52:0/1973281202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be8004280 msgr2=0x7f3be800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f3c01705640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be8004280 0x7f3be800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f3c01705640 1 -- 172.21.15.52:0/1973281202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc0b2680 con 0x7f3be80037a0 2024-05-11T01:01:12.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f3c02707640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be8004280 0x7f3be800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:12.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f3c02707640 1 -- 172.21.15.52:0/1973281202 reap_dead start 2024-05-11T01:01:12.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc7b2d9d640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc79c005370 0x7fc79c0057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f4279ffb640 1 -- 172.21.15.52:0/291129204 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427000b040 con 0x7f427c15b4c0 2024-05-11T01:01:12.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f4279ffb640 1 -- 172.21.15.52:0/291129204 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4270031710 con 0x7f427c15b4c0 2024-05-11T01:01:12.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f4279ffb640 1 -- 172.21.15.52:0/291129204 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270031ac0 con 0x7f427c15b4c0 2024-05-11T01:01:12.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fb88effd640 1 -- 172.21.15.52:0/3410336006 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c033070 con 0x7fb8880e2d70 2024-05-11T01:01:12.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fb88effd640 1 -- 172.21.15.52:0/3410336006 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb87c02f870 con 0x7fb8880e2d70 2024-05-11T01:01:12.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fb88effd640 1 -- 172.21.15.52:0/3410336006 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c02fc40 con 0x7fb8880e2d70 2024-05-11T01:01:12.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7ff62699e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff6200d1500 0x7ff6200cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7ff62699e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6200d36f0 0x7ff6200d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7ff62699e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6200d0460 con 0x7ff62015f580 2024-05-11T01:01:12.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7ff62699e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff6200d0740 con 0x7ff6200d1500 2024-05-11T01:01:12.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7ff62699e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff6200d0a20 con 0x7ff6200d36f0 2024-05-11T01:01:12.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc7b1d9b640 1 -- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc79c005370 msgr2=0x7fc79c0057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc7b1d9b640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc79c005370 0x7fc79c0057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc7b1d9b640 1 -- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc79c004bd0 msgr2=0x7fc79c0037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc7b1d9b640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc79c004bd0 0x7fc79c0037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc7b1d9b640 1 -- 172.21.15.52:0/1495491018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac006200 con 0x7fc79c003dc0 2024-05-11T01:01:12.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc7b259c640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc79c004bd0 0x7fc79c0037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:12.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f537d872640 1 -- 172.21.15.52:0/1216148775 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53780b1260 msgr2=0x7f53780c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7f537d872640 1 --2- 172.21.15.52:0/1216148775 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53780b1260 0x7f53780c7690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53780743e0 tx=0x7f536802ea10 comp rx=0 tx=0).stop 2024-05-11T01:01:12.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc16d33e640 1 -- 172.21.15.52:0/2414099383 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680b2070 con 0x7fc1600078d0 2024-05-11T01:01:12.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc16d33e640 1 -- 172.21.15.52:0/2414099383 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1680ab7d0 con 0x7fc1600078d0 2024-05-11T01:01:12.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.194+0000 7fc16d33e640 1 -- 172.21.15.52:0/2414099383 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680abc40 con 0x7fc1600078d0 2024-05-11T01:01:12.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7ff61ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff62015f580 0x7ff6200d56a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:52912/0 (socket says 172.21.15.52:52912) 2024-05-11T01:01:12.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7ff61ffff640 1 -- 172.21.15.52:0/3533676188 learned_addr learned my addr 172.21.15.52:0/3533676188 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:12.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7ff624f3b640 1 --2- 172.21.15.52:0/3533676188 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6200d36f0 0x7ff6200d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7ff61f7fe640 1 --2- 172.21.15.52:0/3533676188 >> v2:172.21.15.119:3300/0 conn(0x7ff6200d1500 0x7ff6200cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7ff61f7fe640 1 -- 172.21.15.52:0/3533676188 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6200d36f0 msgr2=0x7ff6200d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:12.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7ff61f7fe640 1 --2- 172.21.15.52:0/3533676188 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6200d36f0 0x7ff6200d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7ff61f7fe640 1 -- 172.21.15.52:0/3533676188 >> v1:172.21.15.119:6789/0 conn(0x7ff62015f580 legacy=0x7ff6200d56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7ff61f7fe640 1 -- 172.21.15.52:0/3533676188 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6200d0d00 con 0x7ff6200d1500 2024-05-11T01:01:12.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f3c01705640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 0x7f3be8003c20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0018b0 tx=0x7f3be8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69771a226ecee9e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f42819dd640 1 -- 172.21.15.52:0/291129204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f427c15b4c0 msgr2=0x7f427c0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f42819dd640 1 --2- 172.21.15.52:0/291129204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f427c15b4c0 0x7f427c0729c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f427000a940 tx=0x7f427002c670 comp rx=0 tx=0).stop 2024-05-11T01:01:12.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f537d872640 1 -- 172.21.15.52:0/1216148775 shutdown_connections 2024-05-11T01:01:12.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f537d872640 1 --2- 172.21.15.52:0/1216148775 >> v2:172.21.15.52:3300/0 conn(0x7f53780702a0 0x7f53780d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f537d872640 1 --2- 172.21.15.52:0/1216148775 >> v2:172.21.15.119:3300/0 conn(0x7f5378160730 0x7f537806fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f42819dd640 1 -- 172.21.15.52:0/291129204 shutdown_connections 2024-05-11T01:01:12.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f42819dd640 1 --2- 172.21.15.52:0/291129204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c072f00 0x7f427c06e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7fc16f342640 1 -- 172.21.15.52:0/2414099383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 msgr2=0x7fc1600edd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7fc16f342640 1 --2- 172.21.15.52:0/2414099383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 0x7fc1600edd80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc16802e140 tx=0x7fc1680a6740 comp rx=0 tx=0).stop 2024-05-11T01:01:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f537d872640 1 --2- 172.21.15.52:0/1216148775 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f53780b1260 0x7f53780c7690 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f537d872640 1 -- 172.21.15.52:0/1216148775 >> 172.21.15.52:0/1216148775 conn(0x7f53780b3a10 msgr2=0x7f53780b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f42819dd640 1 --2- 172.21.15.52:0/291129204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f427c15b4c0 0x7f427c0729c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec001500 con 0x7f3be80037a0 2024-05-11T01:01:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bec02e070 con 0x7f3be80037a0 2024-05-11T01:01:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f3c00f04640 1 -- 172.21.15.52:0/1973281202 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec02a9a0 con 0x7f3be80037a0 2024-05-11T01:01:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.195+0000 7f42819dd640 1 -- 172.21.15.52:0/291129204 >> 172.21.15.52:0/291129204 conn(0x7f427c0b2a00 msgr2=0x7f427c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fc7b1d9b640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 0x7fc79c004240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c190 tx=0x7fc79c004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6adb8c393d1f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fc16f342640 1 -- 172.21.15.52:0/2414099383 shutdown_connections 2024-05-11T01:01:12.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fc16f342640 1 --2- 172.21.15.52:0/2414099383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 0x7fc1600edd80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fc16f342640 1 -- 172.21.15.52:0/2414099383 >> 172.21.15.52:0/2414099383 conn(0x7fc1600559d0 msgr2=0x7fc160055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7f42819dd640 1 -- 172.21.15.52:0/291129204 shutdown_connections 2024-05-11T01:01:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fc16f342640 1 -- 172.21.15.52:0/2414099383 shutdown_connections 2024-05-11T01:01:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fc16f342640 1 -- 172.21.15.52:0/2414099383 wait complete. 2024-05-11T01:01:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8036070 con 0x7fc79c003dc0 2024-05-11T01:01:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7f42819dd640 1 -- 172.21.15.52:0/291129204 wait complete. 2024-05-11T01:01:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a8007270 con 0x7fc79c003dc0 2024-05-11T01:01:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fc7b159a640 1 -- 172.21.15.52:0/1495491018 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8007cc0 con 0x7fc79c003dc0 2024-05-11T01:01:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7f55c9d58640 1 Processor -- start 2024-05-11T01:01:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fb89557b640 1 -- 172.21.15.52:0/3410336006 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 msgr2=0x7fb8880e31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fb89557b640 1 --2- 172.21.15.52:0/3410336006 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880e31d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00b160 tx=0x7fb87c02aa10 comp rx=0 tx=0).stop 2024-05-11T01:01:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fb89557b640 1 -- 172.21.15.52:0/3410336006 shutdown_connections 2024-05-11T01:01:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fb89557b640 1 --2- 172.21.15.52:0/3410336006 >> v2:172.21.15.119:3300/0 conn(0x7fb888006ad0 0x7fb888006f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fb89557b640 1 --2- 172.21.15.52:0/3410336006 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880e31d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fb89557b640 1 -- 172.21.15.52:0/3410336006 >> 172.21.15.52:0/3410336006 conn(0x7fb88800a3f0 msgr2=0x7fb88800a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7f537d872640 1 -- 172.21.15.52:0/1216148775 shutdown_connections 2024-05-11T01:01:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7f55c9d58640 1 -- start start 2024-05-11T01:01:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7ff61f7fe640 1 --2- 172.21.15.52:0/3533676188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6200d1500 0x7ff6200cfd80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6100097a0 tx=0x7ff61002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc19e3045eaa5f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fb89557b640 1 -- 172.21.15.52:0/3410336006 shutdown_connections 2024-05-11T01:01:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7fb89557b640 1 -- 172.21.15.52:0/3410336006 wait complete. 2024-05-11T01:01:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.196+0000 7f537d872640 1 -- 172.21.15.52:0/1216148775 wait complete. 2024-05-11T01:01:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7fc7b892b640 1 -- 172.21.15.52:0/1495491018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 msgr2=0x7fc79c004240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7f3c02f08640 1 -- 172.21.15.52:0/1973281202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 msgr2=0x7f3be8003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7f3c02f08640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 0x7f3be8003c20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0018b0 tx=0x7f3be8005830 comp rx=0 tx=0).stop 2024-05-11T01:01:12.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7fc7b892b640 1 --2- 172.21.15.52:0/1495491018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 0x7fc79c004240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c190 tx=0x7fc79c004820 comp rx=0 tx=0).stop 2024-05-11T01:01:12.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7fc7b2d9d640 1 -- 172.21.15.52:0/1495491018 reap_dead start 2024-05-11T01:01:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7f55c9d58640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc0e12a0 0x7f55bc0e47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7fc7b892b640 1 -- 172.21.15.52:0/1495491018 shutdown_connections 2024-05-11T01:01:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7fc7b892b640 1 -- 172.21.15.52:0/1495491018 >> 172.21.15.52:0/1495491018 conn(0x7fc7ac009fa0 msgr2=0x7fc7ac00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7f3c02f08640 1 -- 172.21.15.52:0/1973281202 shutdown_connections 2024-05-11T01:01:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7f3c02f08640 1 --2- 172.21.15.52:0/1973281202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 0x7f3be8003c20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7f3c02f08640 1 -- 172.21.15.52:0/1973281202 >> 172.21.15.52:0/1973281202 conn(0x7f3bfc075830 msgr2=0x7f3bfc075c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7f55c9d58640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55bc0e4d20 0x7f55bc04e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.197+0000 7fc7b892b640 1 -- 172.21.15.52:0/1495491018 shutdown_connections 2024-05-11T01:01:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f42819dd640 1 Processor -- start 2024-05-11T01:01:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7fc7b892b640 1 -- 172.21.15.52:0/1495491018 wait complete. 2024-05-11T01:01:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f3c02f08640 1 -- 172.21.15.52:0/1973281202 shutdown_connections 2024-05-11T01:01:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c9d58640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55bc050300 0x7f55bc0df3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c37fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc0e12a0 0x7f55bc0e47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7fc16f342640 1 Processor -- start 2024-05-11T01:01:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c37fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc0e12a0 0x7f55bc0e47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41084/0 (socket says 172.21.15.52:41084) 2024-05-11T01:01:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7fc16f342640 1 -- start start 2024-05-11T01:01:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c9d58640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc057690 con 0x7f55bc0e4d20 2024-05-11T01:01:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c9d58640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc057810 con 0x7f55bc050300 2024-05-11T01:01:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c9d58640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc0e1d90 con 0x7f55bc0e12a0 2024-05-11T01:01:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c37fe640 1 -- 172.21.15.52:0/52162609 learned_addr learned my addr 172.21.15.52:0/52162609 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c3fff640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55bc050300 0x7f55bc0df3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f42819dd640 1 -- start start 2024-05-11T01:01:12.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f3c02f08640 1 -- 172.21.15.52:0/1973281202 wait complete. 2024-05-11T01:01:12.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c2ffd640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55bc0e4d20 0x7f55bc04e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f537d872640 1 Processor -- start 2024-05-11T01:01:12.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c3fff640 1 -- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc0e12a0 msgr2=0x7f55bc0e47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c3fff640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc0e12a0 0x7f55bc0e47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c3fff640 1 -- 172.21.15.52:0/52162609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55bc0e4d20 msgr2=0x7f55bc04e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c3fff640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55bc0e4d20 0x7f55bc04e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f55c3fff640 1 -- 172.21.15.52:0/52162609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc04f560 con 0x7f55bc050300 2024-05-11T01:01:12.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fb89557b640 1 Processor -- start 2024-05-11T01:01:12.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.198+0000 7f537d872640 1 -- start start 2024-05-11T01:01:12.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7ff61effd640 1 -- 172.21.15.52:0/3533676188 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610037070 con 0x7ff6200d1500 2024-05-11T01:01:12.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fc16f342640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 0x7fc16017cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7ff61effd640 1 -- 172.21.15.52:0/3533676188 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff610032440 con 0x7ff6200d1500 2024-05-11T01:01:12.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fc16f342640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc16003e830 0x7fc1600dfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fc16e340640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 0x7fc16017cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fc16e340640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 0x7fc16017cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35190/0 (socket says 172.21.15.52:35190) 2024-05-11T01:01:12.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fc16e340640 1 -- 172.21.15.52:0/837436109 learned_addr learned my addr 172.21.15.52:0/837436109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fc16f342640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1600e30b0 0x7fc1600da8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fc16f342640 1 -- 172.21.15.52:0/837436109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1600542f0 con 0x7fc1600078d0 2024-05-11T01:01:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fc16f342640 1 -- 172.21.15.52:0/837436109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1600e3490 con 0x7fc1600e30b0 2024-05-11T01:01:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fc16f342640 1 -- 172.21.15.52:0/837436109 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc160054470 con 0x7fc16003e830 2024-05-11T01:01:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f42819dd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c06ee20 0x7f427c1f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc16eb41640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1600e30b0 0x7fc1600da8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f42819dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f427c072f00 0x7f427c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc16e340640 1 -- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc16003e830 msgr2=0x7fc1600dfe40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T01:01:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7ff62699e640 1 -- 172.21.15.52:0/3533676188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6200d1500 msgr2=0x7ff6200cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7ff62699e640 1 --2- 172.21.15.52:0/3533676188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6200d1500 0x7ff6200cfd80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6100097a0 tx=0x7ff61002a180 comp rx=0 tx=0).stop 2024-05-11T01:01:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc16e340640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc16003e830 0x7fc1600dfe40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc16e340640 1 -- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1600e30b0 msgr2=0x7fc1600da8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc16e340640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1600e30b0 0x7fc1600da8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc16e340640 1 -- 172.21.15.52:0/837436109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1600e0380 con 0x7fc1600078d0 2024-05-11T01:01:12.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7fb89557b640 1 -- start start 2024-05-11T01:01:12.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fb89557b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb888006ad0 0x7fb888037250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f427affd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c06ee20 0x7f427c1f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f427affd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c06ee20 0x7f427c1f16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41098/0 (socket says 172.21.15.52:41098) 2024-05-11T01:01:12.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f427affd640 1 -- 172.21.15.52:0/2551974345 learned_addr learned my addr 172.21.15.52:0/2551974345 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7f55c3fff640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55bc050300 0x7f55bc0df3a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55c4064020 tx=0x7f55c4076a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7f55affff640 1 -- 172.21.15.52:0/52162609 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c406dc50 con 0x7f55bc050300 2024-05-11T01:01:12.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc04ed80 con 0x7f55bc050300 2024-05-11T01:01:12.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.199+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc04f820 con 0x7f55bc050300 2024-05-11T01:01:12.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f537d872640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53780702a0 0x7f53781579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fb89557b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8880df210 0x7fb8880cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fb89557b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fb89557b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb8880065b0 con 0x7fb888006ad0 2024-05-11T01:01:12.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fb89557b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb8880062d0 con 0x7fb8880df210 2024-05-11T01:01:12.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fb89557b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb8880df5f0 con 0x7fb8880e2d70 2024-05-11T01:01:12.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f537d872640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53780b1260 0x7f53781f9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f537d872640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5378160730 0x7f53780afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f537d872640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f53780b1640 con 0x7f53780702a0 2024-05-11T01:01:12.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f537d872640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5378160b90 con 0x7f53780b1260 2024-05-11T01:01:12.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f537d872640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f53780b17c0 con 0x7f5378160730 2024-05-11T01:01:12.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f5376ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53780702a0 0x7f53781579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f5376ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53780702a0 0x7f53781579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35192/0 (socket says 172.21.15.52:35192) 2024-05-11T01:01:12.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f5376ffd640 1 -- 172.21.15.52:0/3563344160 learned_addr learned my addr 172.21.15.52:0/3563344160 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc7b892b640 1 Processor -- start 2024-05-11T01:01:12.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f55affff640 1 -- 172.21.15.52:0/52162609 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c406ddf0 con 0x7f55bc050300 2024-05-11T01:01:12.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f55affff640 1 -- 172.21.15.52:0/52162609 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40741c0 con 0x7f55bc050300 2024-05-11T01:01:12.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc7b892b640 1 -- start start 2024-05-11T01:01:12.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc16e340640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 0x7fc16017cdd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c500 tx=0x7fc16400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fb894d7a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fb894d7a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880d2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41108/0 (socket says 172.21.15.52:41108) 2024-05-11T01:01:12.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fb894d7a640 1 -- 172.21.15.52:0/3354606106 learned_addr learned my addr 172.21.15.52:0/3354606106 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7fc15effd640 1 -- 172.21.15.52:0/837436109 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164007480 con 0x7fc1600078d0 2024-05-11T01:01:12.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f53777fe640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5378160730 0x7f53780afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7f427a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f427c072f00 0x7f427c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f5376ffd640 1 -- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5378160730 msgr2=0x7f53780afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f5376ffd640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5378160730 0x7f53780afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f5376ffd640 1 -- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53780b1260 msgr2=0x7f53781f9260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T01:01:12.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f5376ffd640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53780b1260 0x7f53781f9260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f5376ffd640 1 -- 172.21.15.52:0/3563344160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53780b75e0 con 0x7f53780702a0 2024-05-11T01:01:12.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f42819dd640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f427c15b4c0 0x7f427c14a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f42819dd640 1 -- 172.21.15.52:0/2551974345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f427c0b50d0 con 0x7f427c072f00 2024-05-11T01:01:12.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f42819dd640 1 -- 172.21.15.52:0/2551974345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f427c06f270 con 0x7f427c15b4c0 2024-05-11T01:01:12.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f42819dd640 1 -- 172.21.15.52:0/2551974345 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f427c0b5250 con 0x7f427c06ee20 2024-05-11T01:01:12.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7fb894d7a640 1 -- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8880df210 msgr2=0x7fb8880cde20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:01:12.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7fb894d7a640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8880df210 0x7fb8880cde20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7fb894d7a640 1 -- 172.21.15.52:0/3354606106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb888006ad0 msgr2=0x7fb888037250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:01:12.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7fb894d7a640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb888006ad0 0x7fb888037250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7fb894d7a640 1 -- 172.21.15.52:0/3354606106 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb888049960 con 0x7fb8880e2d70 2024-05-11T01:01:12.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7ff62699e640 1 -- 172.21.15.52:0/3533676188 shutdown_connections 2024-05-11T01:01:12.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7ff62699e640 1 --2- 172.21.15.52:0/3533676188 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6200d36f0 0x7ff6200d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7ff62699e640 1 --2- 172.21.15.52:0/3533676188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6200d1500 0x7ff6200cfd80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.200+0000 7ff62699e640 1 -- 172.21.15.52:0/3533676188 >> 172.21.15.52:0/3533676188 conn(0x7ff62008eb80 msgr2=0x7ff62009e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7ff62699e640 1 -- 172.21.15.52:0/3533676188 shutdown_connections 2024-05-11T01:01:12.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7fc7b892b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 0x7fc7ac0d67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f53777fe640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5378160730 0x7f53780afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:12.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f5376ffd640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53780702a0 0x7f53781579b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f536800bb00 tx=0x7f53680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f3c02f08640 1 Processor -- start 2024-05-11T01:01:12.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f3c02f08640 1 -- start start 2024-05-11T01:01:12.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7f427b7fe640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f427c15b4c0 0x7f427c14a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f427a7fc640 1 -- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c06ee20 msgr2=0x7f427c1f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f427a7fc640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c06ee20 0x7f427c1f16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f427a7fc640 1 -- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f427c15b4c0 msgr2=0x7f427c14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f427a7fc640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f427c15b4c0 0x7f427c14a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f427a7fc640 1 -- 172.21.15.52:0/2551974345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c0b4660 con 0x7f427c072f00 2024-05-11T01:01:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc0057d0 con 0x7f55bc050300 2024-05-11T01:01:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7fc16f342640 1 -- 172.21.15.52:0/837436109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1600dadf0 con 0x7fc1600078d0 2024-05-11T01:01:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7fc16f342640 1 -- 172.21.15.52:0/837436109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1600db370 con 0x7fc1600078d0 2024-05-11T01:01:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7fc15effd640 1 -- 172.21.15.52:0/837436109 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc164007620 con 0x7fc1600078d0 2024-05-11T01:01:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7fc15effd640 1 -- 172.21.15.52:0/837436109 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164004a20 con 0x7fc1600078d0 2024-05-11T01:01:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7fc7b892b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac1743f0 0x7fc7ac0c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7fc7b892b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7ac0356a0 0x7fc7ac035b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f535ffff640 1 -- 172.21.15.52:0/3563344160 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368037040 con 0x7f53780702a0 2024-05-11T01:01:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f535ffff640 1 -- 172.21.15.52:0/3563344160 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536803c450 con 0x7f53780702a0 2024-05-11T01:01:12.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f3c02f08640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 0x7f3bfc0a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f535ffff640 1 -- 172.21.15.52:0/3563344160 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536803c840 con 0x7f53780702a0 2024-05-11T01:01:12.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53780b6540 con 0x7f53780702a0 2024-05-11T01:01:12.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53780b6b00 con 0x7f53780702a0 2024-05-11T01:01:12.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.202+0000 7f427a7fc640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f427c072f00 0x7f427c0c5c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4274007e60 tx=0x7f427400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f55affff640 1 -- 172.21.15.52:0/52162609 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f55c4074440 con 0x7f55bc050300 2024-05-11T01:01:12.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f3c02f08640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3bfc160030 0x7f3bfc1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fb894d7a640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880d2f90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb8900521a0 tx=0x7fb890074530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fc7b1d9b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac1743f0 0x7fc7ac0c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fc7b1d9b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac1743f0 0x7fc7ac0c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41118/0 (socket says 172.21.15.52:41118) 2024-05-11T01:01:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fc7b1d9b640 1 -- 172.21.15.52:0/2114106593 learned_addr learned my addr 172.21.15.52:0/2114106593 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac037a80 con 0x7fc79c003dc0 2024-05-11T01:01:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fb88cff9640 1 -- 172.21.15.52:0/3354606106 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89006c480 con 0x7fb8880e2d70 2024-05-11T01:01:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fc7b2d9d640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7ac0356a0 0x7fc7ac035b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac037900 con 0x7fc7ac0356a0 2024-05-11T01:01:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac0df5f0 con 0x7fc7ac1743f0 2024-05-11T01:01:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fb88cff9640 1 -- 172.21.15.52:0/3354606106 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89006c620 con 0x7fb8880e2d70 2024-05-11T01:01:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fb89557b640 1 -- 172.21.15.52:0/3354606106 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb888049c40 con 0x7fb8880e2d70 2024-05-11T01:01:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f3c01705640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3bfc160030 0x7f3bfc1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f3c01705640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3bfc160030 0x7f3bfc1604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41130/0 (socket says 172.21.15.52:41130) 2024-05-11T01:01:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fb89557b640 1 -- 172.21.15.52:0/3354606106 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88804a200 con 0x7fb8880e2d70 2024-05-11T01:01:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7fc7b259c640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 0x7fc7ac0d67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f3c02f08640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bfc0ca6c0 0x7f3bfc0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f3c02f08640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc06eb90 con 0x7f3be80037a0 2024-05-11T01:01:12.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f3c02f08640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc06ea10 con 0x7f3bfc0ca6c0 2024-05-11T01:01:12.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f3c02f08640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3bfc06ed10 con 0x7f3bfc160030 2024-05-11T01:01:12.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7f3c01705640 1 -- 172.21.15.52:0/277831105 learned_addr learned my addr 172.21.15.52:0/277831105 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.201+0000 7ff62699e640 1 -- 172.21.15.52:0/3533676188 wait complete. 2024-05-11T01:01:12.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.203+0000 7ff62699e640 1 Processor -- start 2024-05-11T01:01:12.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7fc7b2d9d640 1 -- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac1743f0 msgr2=0x7fc7ac0c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7fc7b2d9d640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac1743f0 0x7fc7ac0c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7fc7b2d9d640 1 -- 172.21.15.52:0/2114106593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 msgr2=0x7fc7ac0d67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f3c02707640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bfc0ca6c0 0x7f3bfc0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7fc7b2d9d640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 0x7fc7ac0d67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f3c01705640 1 -- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bfc0ca6c0 msgr2=0x7f3bfc0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f3c01705640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bfc0ca6c0 0x7f3bfc0cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f3c01705640 1 -- 172.21.15.52:0/277831105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 msgr2=0x7f3bfc0a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f3c01705640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 0x7f3bfc0a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f3c01705640 1 -- 172.21.15.52:0/277831105 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc15d8b0 con 0x7f3bfc160030 2024-05-11T01:01:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7ff62699e640 1 -- start start 2024-05-11T01:01:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7fc7b2d9d640 1 -- 172.21.15.52:0/2114106593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac0d7f30 con 0x7fc7ac0356a0 2024-05-11T01:01:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7fc7b259c640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 0x7fc7ac0d67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7fb88cff9640 1 -- 172.21.15.52:0/3354606106 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89006c810 con 0x7fb8880e2d70 2024-05-11T01:01:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f55affff640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f55a80763a0 0x7f55a8078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7fc7b2d9d640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7ac0356a0 0x7fc7ac035b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7b407bf10 tx=0x7fc7b40640c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7f55affff640 1 -- 172.21.15.52:0/52162609 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f55c4113200 con 0x7f55bc050300 2024-05-11T01:01:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7f55c37fe640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f55a80763a0 0x7f55a8078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7fc15cff9640 1 -- 172.21.15.52:0/837436109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc150005290 con 0x7fc1600078d0 2024-05-11T01:01:12.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f3c01705640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3bfc160030 0x7f3bfc1604b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0019d0 tx=0x7f3bec0012f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7f3bf27fc640 1 -- 172.21.15.52:0/277831105 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec043070 con 0x7f3bfc160030 2024-05-11T01:01:12.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7f3bf27fc640 1 -- 172.21.15.52:0/277831105 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bec003a00 con 0x7f3bfc160030 2024-05-11T01:01:12.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7f3bf27fc640 1 -- 172.21.15.52:0/277831105 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec001500 con 0x7f3bfc160030 2024-05-11T01:01:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bfc0a2cd0 con 0x7f3bfc160030 2024-05-11T01:01:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bfc151120 con 0x7f3bfc160030 2024-05-11T01:01:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f4263fff640 1 -- 172.21.15.52:0/2551974345 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274014620 con 0x7f427c072f00 2024-05-11T01:01:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f4263fff640 1 -- 172.21.15.52:0/2551974345 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4274019070 con 0x7f427c072f00 2024-05-11T01:01:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.204+0000 7f4263fff640 1 -- 172.21.15.52:0/2551974345 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42740149e0 con 0x7f427c072f00 2024-05-11T01:01:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7fb89557b640 1 -- 172.21.15.52:0/3354606106 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb888008e10 con 0x7fb8880e2d70 2024-05-11T01:01:12.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7f42819dd640 1 -- 172.21.15.52:0/2551974345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c0b4940 con 0x7f427c072f00 2024-05-11T01:01:12.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7f42819dd640 1 -- 172.21.15.52:0/2551974345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c1f0ee0 con 0x7f427c072f00 2024-05-11T01:01:12.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7ff62699e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6200d1500 0x7ff6200caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.205+0000 7ff62699e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6200d36f0 0x7ff620159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7fc15effd640 1 -- 172.21.15.52:0/837436109 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc16401a020 con 0x7fc1600078d0 2024-05-11T01:01:12.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff62699e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015f580 0x7ff620159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff62699e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6200c8cc0 con 0x7ff6200d36f0 2024-05-11T01:01:12.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff62699e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff62015f960 con 0x7ff62015f580 2024-05-11T01:01:12.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff62699e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff6200c8e40 con 0x7ff6200d1500 2024-05-11T01:01:12.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff61f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6200d36f0 0x7ff620159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff61f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6200d36f0 0x7ff620159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35220/0 (socket says 172.21.15.52:35220) 2024-05-11T01:01:12.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff61f7fe640 1 -- 172.21.15.52:0/643903303 learned_addr learned my addr 172.21.15.52:0/643903303 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:12.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7fb88cff9640 1 -- 172.21.15.52:0/3354606106 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb89006ca70 con 0x7fb8880e2d70 2024-05-11T01:01:12.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff624f3b640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015f580 0x7ff620159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7f55c37fe640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f55a80763a0 0x7f55a8078860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009c00 tx=0x7f55b8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7fc15effd640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc14c0762d0 0x7fc14c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff61f7fe640 1 -- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6200d1500 msgr2=0x7ff6200caca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T01:01:12.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff61f7fe640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6200d1500 0x7ff6200caca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff61f7fe640 1 -- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015f580 msgr2=0x7ff620159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff61f7fe640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015f580 0x7ff620159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff61f7fe640 1 -- 172.21.15.52:0/643903303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6200cc7b0 con 0x7ff6200d36f0 2024-05-11T01:01:12.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7ff624f3b640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015f580 0x7ff620159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:12.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7fc15effd640 1 -- 172.21.15.52:0/837436109 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc164096e40 con 0x7fc1600078d0 2024-05-11T01:01:12.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7f535ffff640 1 -- 172.21.15.52:0/3563344160 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f536803a070 con 0x7f53780702a0 2024-05-11T01:01:12.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.206+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5378069780 con 0x7f53780702a0 2024-05-11T01:01:12.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7ff61f7fe640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6200d36f0 0x7ff620159450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6100098c0 tx=0x7ff6100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be4005290 con 0x7f3bfc160030 2024-05-11T01:01:12.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7fc16db3f640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc14c0762d0 0x7fc14c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7fb88cff9640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb8740763a0 0x7fb874078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7f535ffff640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5364076320 0x7f53640787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7fb88cff9640 1 -- 172.21.15.52:0/3354606106 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8901132b0 con 0x7fb8880e2d70 2024-05-11T01:01:12.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7f535ffff640 1 -- 172.21.15.52:0/3563344160 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53680b8cf0 con 0x7f53780702a0 2024-05-11T01:01:12.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7fb88ffff640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb8740763a0 0x7fb874078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.207+0000 7f53767fc640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5364076320 0x7f53640787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.208+0000 7fc16db3f640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc14c0762d0 0x7fc14c078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc158001220 tx=0x7fc158007870 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.208+0000 7fb88ffff640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb8740763a0 0x7fb874078860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb884001590 tx=0x7fb884007c90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.208+0000 7f53767fc640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5364076320 0x7f53640787e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53780b0dc0 tx=0x7f536c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.208+0000 7f42819dd640 1 -- 172.21.15.52:0/2551974345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427c069780 con 0x7f427c072f00 2024-05-11T01:01:12.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.208+0000 7f3bf27fc640 1 -- 172.21.15.52:0/277831105 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3bec03e470 con 0x7f3bfc160030 2024-05-11T01:01:12.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7f3bf27fc640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3be00763d0 0x7f3be0078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7f3bf27fc640 1 -- 172.21.15.52:0/277831105 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3bec0b7670 con 0x7f3bfc160030 2024-05-11T01:01:12.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7f3c01f06640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3be00763d0 0x7f3be0078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/2114106593 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4009690 con 0x7fc7ac0356a0 2024-05-11T01:01:12.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/2114106593 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b40748a0 con 0x7fc7ac0356a0 2024-05-11T01:01:12.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac0d6e90 con 0x7fc7ac0356a0 2024-05-11T01:01:12.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7f55affff640 1 -- 172.21.15.52:0/52162609 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f55c4090030 con 0x7f55bc050300 2024-05-11T01:01:12.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/2114106593 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b406c480 con 0x7fc7ac0356a0 2024-05-11T01:01:12.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac0d7450 con 0x7fc7ac0356a0 2024-05-11T01:01:12.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.209+0000 7f3c01f06640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3be00763d0 0x7f3be0078890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8006a90 tx=0x7f3bf8006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.211+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ac005380 con 0x7fc7ac0356a0 2024-05-11T01:01:12.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.211+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/2114106593 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc7b406c620 con 0x7fc7ac0356a0 2024-05-11T01:01:12.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.212+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7940762d0 0x7fc794078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.212+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/2114106593 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7b4112fd0 con 0x7fc7ac0356a0 2024-05-11T01:01:12.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.212+0000 7fb88cff9640 1 -- 172.21.15.52:0/3354606106 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb8900dcbf0 con 0x7fb8880e2d70 2024-05-11T01:01:12.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.212+0000 7fc7b259c640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7940762d0 0x7fc794078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.213+0000 7fc7b259c640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7940762d0 0x7fc794078790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009d60 tx=0x7fc7a4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.213+0000 7fc15effd640 1 -- 172.21.15.52:0/837436109 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc164060790 con 0x7fc1600078d0 2024-05-11T01:01:12.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.217+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/2114106593 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc7b40b4020 con 0x7fc7ac0356a0 2024-05-11T01:01:12.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.217+0000 7f3bf27fc640 1 -- 172.21.15.52:0/277831105 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3bfc15d8b0 con 0x7f3bfc160030 2024-05-11T01:01:12.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.219+0000 7ff61cff9640 1 -- 172.21.15.52:0/643903303 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610040020 con 0x7ff6200d36f0 2024-05-11T01:01:12.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.219+0000 7ff62699e640 1 -- 172.21.15.52:0/643903303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6200cca90 con 0x7ff6200d36f0 2024-05-11T01:01:12.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.219+0000 7ff62699e640 1 -- 172.21.15.52:0/643903303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6200cd050 con 0x7ff6200d36f0 2024-05-11T01:01:12.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.221+0000 7ff6067fc640 1 -- 172.21.15.52:0/643903303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff620069780 con 0x7ff6200d36f0 2024-05-11T01:01:12.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.220+0000 7f4263fff640 1 -- 172.21.15.52:0/2551974345 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4274014b80 con 0x7f427c072f00 2024-05-11T01:01:12.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.220+0000 7f4263fff640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f42680763a0 0x7f4268078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.221+0000 7f4263fff640 1 -- 172.21.15.52:0/2551974345 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f42740988e0 con 0x7f427c072f00 2024-05-11T01:01:12.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.221+0000 7f427affd640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f42680763a0 0x7f4268078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.218+0000 7f535ffff640 1 -- 172.21.15.52:0/3563344160 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f53680827c0 con 0x7f53780702a0 2024-05-11T01:01:12.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.222+0000 7f427affd640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f42680763a0 0x7f4268078860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f427002c6b0 tx=0x7f427000c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.222+0000 7ff61cff9640 1 -- 172.21.15.52:0/643903303 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff610038040 con 0x7ff6200d36f0 2024-05-11T01:01:12.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.222+0000 7ff61cff9640 1 -- 172.21.15.52:0/643903303 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610037070 con 0x7ff6200d36f0 2024-05-11T01:01:12.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.226+0000 7f4263fff640 1 -- 172.21.15.52:0/2551974345 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f42740622b0 con 0x7f427c072f00 2024-05-11T01:01:12.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.228+0000 7ff61cff9640 1 -- 172.21.15.52:0/643903303 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff610043020 con 0x7ff6200d36f0 2024-05-11T01:01:12.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.229+0000 7ff61cff9640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff60c0762d0 0x7ff60c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:12.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.229+0000 7ff61cff9640 1 -- 172.21.15.52:0/643903303 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff6100b7ff0 con 0x7ff6200d36f0 2024-05-11T01:01:12.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.230+0000 7ff61ffff640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff60c0762d0 0x7ff60c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:12.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.236+0000 7ff61cff9640 1 -- 172.21.15.52:0/643903303 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff61004a030 con 0x7ff6200d36f0 2024-05-11T01:01:12.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.246+0000 7ff61ffff640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff60c0762d0 0x7ff60c078790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6140014c0 tx=0x7ff614007a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.661+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f55bc057470 con 0x7f55bc050300 2024-05-11T01:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.661+0000 7f55affff640 1 -- 172.21.15.52:0/52162609 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55c40dcbd0 con 0x7f55bc050300 2024-05-11T01:01:12.663 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:12.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.663+0000 7fb89557b640 1 -- 172.21.15.52:0/3354606106 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb8880057d0 con 0x7fb8880e2d70 2024-05-11T01:01:12.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.663+0000 7fb88cff9640 1 -- 172.21.15.52:0/3354606106 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb8900dc4f0 con 0x7fb8880e2d70 2024-05-11T01:01:12.664 INFO:teuthology.orchestra.run.smithi052.stdout:55834574853 2024-05-11T01:01:12.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.665+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f55a80763a0 msgr2=0x7f55a8078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.665+0000 7f55c9d58640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f55a80763a0 0x7f55a8078860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009c00 tx=0x7f55b8009190 comp rx=0 tx=0).stop 2024-05-11T01:01:12.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.666+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55bc050300 msgr2=0x7f55bc0df3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.666+0000 7f55c9d58640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55bc050300 0x7f55bc0df3a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55c4064020 tx=0x7f55c4076a80 comp rx=0 tx=0).stop 2024-05-11T01:01:12.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.666+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 shutdown_connections 2024-05-11T01:01:12.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.666+0000 7f55c9d58640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f55a80763a0 0x7f55a8078860 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.666+0000 7f55c9d58640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55bc050300 0x7f55bc0df3a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.666+0000 7f55c9d58640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55bc0e4d20 0x7f55bc04e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.666+0000 7f55c9d58640 1 --2- 172.21.15.52:0/52162609 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55bc0e12a0 0x7f55bc0e47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.666+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 >> 172.21.15.52:0/52162609 conn(0x7f55bc020030 msgr2=0x7f55bc008dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.666+0000 7fb87a7fc640 1 -- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb8740763a0 msgr2=0x7fb874078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fb87a7fc640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb8740763a0 0x7fb874078860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb884001590 tx=0x7fb884007c90 comp rx=0 tx=0).stop 2024-05-11T01:01:12.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fb87a7fc640 1 -- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 msgr2=0x7fb8880d2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fb87a7fc640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880d2f90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb8900521a0 tx=0x7fb890074530 comp rx=0 tx=0).stop 2024-05-11T01:01:12.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 shutdown_connections 2024-05-11T01:01:12.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7f55c9d58640 1 -- 172.21.15.52:0/52162609 wait complete. 2024-05-11T01:01:12.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fb87a7fc640 1 -- 172.21.15.52:0/3354606106 shutdown_connections 2024-05-11T01:01:12.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fb87a7fc640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb8740763a0 0x7fb874078860 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fb87a7fc640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb8880e2d70 0x7fb8880d2f90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fb87a7fc640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8880df210 0x7fb8880cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fb87a7fc640 1 --2- 172.21.15.52:0/3354606106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb888006ad0 0x7fb888037250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fb87a7fc640 1 -- 172.21.15.52:0/3354606106 >> 172.21.15.52:0/3354606106 conn(0x7fb88800a3f0 msgr2=0x7fb888037ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.667+0000 7fc15cff9640 1 -- 172.21.15.52:0/837436109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc150005580 con 0x7fc1600078d0 2024-05-11T01:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.668+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc7ac052bd0 con 0x7fc7ac0356a0 2024-05-11T01:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.668+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f53780d1690 con 0x7f53780702a0 2024-05-11T01:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.668+0000 7fb87a7fc640 1 -- 172.21.15.52:0/3354606106 shutdown_connections 2024-05-11T01:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.668+0000 7fb87a7fc640 1 -- 172.21.15.52:0/3354606106 wait complete. 2024-05-11T01:01:12.672 INFO:teuthology.orchestra.run.smithi052.stdout:55834574853 2024-05-11T01:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.668+0000 7f535ffff640 1 -- 172.21.15.52:0/3563344160 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f53680827c0 con 0x7f53780702a0 2024-05-11T01:01:12.672 INFO:teuthology.orchestra.run.smithi052.stdout:60129542150 2024-05-11T01:01:12.673 INFO:teuthology.orchestra.run.smithi052.stdout:55834574853 2024-05-11T01:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.669+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/2114106593 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7ac0d7f30 con 0x7fc7ac0356a0 2024-05-11T01:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.671+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3be4005580 con 0x7f3bfc160030 2024-05-11T01:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.668+0000 7fc15effd640 1 -- 172.21.15.52:0/837436109 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc164060090 con 0x7fc1600078d0 2024-05-11T01:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.671+0000 7f3bf27fc640 1 -- 172.21.15.52:0/277831105 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3bec080960 con 0x7f3bfc160030 2024-05-11T01:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5364076320 msgr2=0x7f53640787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7f537d872640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5364076320 0x7f53640787e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53780b0dc0 tx=0x7f536c009190 comp rx=0 tx=0).stop 2024-05-11T01:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53780702a0 msgr2=0x7f53781579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7f537d872640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53780702a0 0x7f53781579b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f536800bb00 tx=0x7f53680046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.675 INFO:teuthology.orchestra.run.smithi052.stdout:55834574853 2024-05-11T01:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 -- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc14c0762d0 msgr2=0x7fc14c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc14c0762d0 0x7fc14c078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc158001220 tx=0x7fc158007870 comp rx=0 tx=0).stop 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 -- 172.21.15.52:0/837436109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 msgr2=0x7fc16017cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 0x7fc16017cdd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c500 tx=0x7fc16400c9c0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 -- 172.21.15.52:0/837436109 shutdown_connections 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc14c0762d0 0x7fc14c078790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1600e30b0 0x7fc1600da8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc16003e830 0x7fc1600dfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 --2- 172.21.15.52:0/837436109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1600078d0 0x7fc16017cdd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 -- 172.21.15.52:0/837436109 >> 172.21.15.52:0/837436109 conn(0x7fc1600559d0 msgr2=0x7fc160005bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.672+0000 7fc15cff9640 1 -- 172.21.15.52:0/837436109 shutdown_connections 2024-05-11T01:01:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.673+0000 7fc15cff9640 1 -- 172.21.15.52:0/837436109 wait complete. 2024-05-11T01:01:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.673+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 shutdown_connections 2024-05-11T01:01:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.673+0000 7f537d872640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5364076320 0x7f53640787e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.673+0000 7f537d872640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5378160730 0x7f53780afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.673+0000 7f537d872640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53780b1260 0x7f53781f9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.673+0000 7f537d872640 1 --2- 172.21.15.52:0/3563344160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53780702a0 0x7f53781579b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.673+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 >> 172.21.15.52:0/3563344160 conn(0x7f53780b3a10 msgr2=0x7f53780d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7940762d0 msgr2=0x7fc794078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7940762d0 0x7fc794078790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009d60 tx=0x7fc7a4008040 comp rx=0 tx=0).stop 2024-05-11T01:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7ac0356a0 msgr2=0x7fc7ac035b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7ac0356a0 0x7fc7ac035b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7b407bf10 tx=0x7fc7b40640c0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 shutdown_connections 2024-05-11T01:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7f537d872640 1 -- 172.21.15.52:0/3563344160 wait complete. 2024-05-11T01:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 shutdown_connections 2024-05-11T01:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7940762d0 0x7fc794078790 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7ac0356a0 0x7fc7ac035b20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7ac1743f0 0x7fc7ac0c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 --2- 172.21.15.52:0/2114106593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc79c003dc0 0x7fc7ac0d67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 >> 172.21.15.52:0/2114106593 conn(0x7fc7ac009fa0 msgr2=0x7fc7ac036580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.675+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3be00763d0 msgr2=0x7f3be0078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.675+0000 7f3c02f08640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3be00763d0 0x7f3be0078890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8006a90 tx=0x7f3bf8006a20 comp rx=0 tx=0).stop 2024-05-11T01:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.675+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3bfc160030 msgr2=0x7f3bfc1604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.675+0000 7f3c02f08640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3bfc160030 0x7f3bfc1604b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0019d0 tx=0x7f3bec0012f0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 shutdown_connections 2024-05-11T01:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.676+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 shutdown_connections 2024-05-11T01:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.676+0000 7f3c02f08640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3be00763d0 0x7f3be0078890 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.676+0000 7f3c02f08640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bfc0ca6c0 0x7f3bfc0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.676+0000 7f3c02f08640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3bfc160030 0x7f3bfc1604b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.676+0000 7f3c02f08640 1 --2- 172.21.15.52:0/277831105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be80037a0 0x7f3bfc0a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.676+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 >> 172.21.15.52:0/277831105 conn(0x7f3bfc075830 msgr2=0x7f3bfc06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.674+0000 7fc7b892b640 1 -- 172.21.15.52:0/2114106593 wait complete. 2024-05-11T01:01:12.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.676+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 shutdown_connections 2024-05-11T01:01:12.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.676+0000 7f3c02f08640 1 -- 172.21.15.52:0/277831105 wait complete. 2024-05-11T01:01:12.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.679+0000 7f42819dd640 1 -- 172.21.15.52:0/2551974345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f427c15eef0 con 0x7f427c072f00 2024-05-11T01:01:12.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.680+0000 7f4263fff640 1 -- 172.21.15.52:0/2551974345 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4274061bb0 con 0x7f427c072f00 2024-05-11T01:01:12.684 INFO:teuthology.orchestra.run.smithi052.stdout:55834574853 2024-05-11T01:01:12.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.683+0000 7ff6067fc640 1 -- 172.21.15.52:0/643903303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff6200c8790 con 0x7ff6200d36f0 2024-05-11T01:01:12.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.684+0000 7ff61cff9640 1 -- 172.21.15.52:0/643903303 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff6100819e0 con 0x7ff6200d36f0 2024-05-11T01:01:12.685 INFO:teuthology.orchestra.run.smithi052.stdout:55834574853 2024-05-11T01:01:12.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 -- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f42680763a0 msgr2=0x7f4268078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f42680763a0 0x7f4268078860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f427002c6b0 tx=0x7f427000c040 comp rx=0 tx=0).stop 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 -- 172.21.15.52:0/2551974345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f427c072f00 msgr2=0x7f427c0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f427c072f00 0x7f427c0c5c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4274007e60 tx=0x7f427400e4b0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 -- 172.21.15.52:0/2551974345 shutdown_connections 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f42680763a0 0x7f4268078860 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f427c15b4c0 0x7f427c14a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f427c072f00 0x7f427c0c5c70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 --2- 172.21.15.52:0/2551974345 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f427c06ee20 0x7f427c1f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 -- 172.21.15.52:0/2551974345 >> 172.21.15.52:0/2551974345 conn(0x7f427c0b2a00 msgr2=0x7f427c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 -- 172.21.15.52:0/2551974345 shutdown_connections 2024-05-11T01:01:12.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.685+0000 7f4261ffb640 1 -- 172.21.15.52:0/2551974345 wait complete. 2024-05-11T01:01:12.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.688+0000 7ff6067fc640 1 -- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff60c0762d0 msgr2=0x7ff60c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.688+0000 7ff6067fc640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff60c0762d0 0x7ff60c078790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6140014c0 tx=0x7ff614007a20 comp rx=0 tx=0).stop 2024-05-11T01:01:12.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.688+0000 7ff6067fc640 1 -- 172.21.15.52:0/643903303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6200d36f0 msgr2=0x7ff620159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:12.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.688+0000 7ff6067fc640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6200d36f0 0x7ff620159450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6100098c0 tx=0x7ff6100046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.688+0000 7ff6067fc640 1 -- 172.21.15.52:0/643903303 shutdown_connections 2024-05-11T01:01:12.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.689+0000 7ff6067fc640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ff60c0762d0 0x7ff60c078790 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6140014c0 tx=0x7ff614007a20 comp rx=0 tx=0).stop 2024-05-11T01:01:12.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.689+0000 7ff6067fc640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff62015f580 0x7ff620159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.689+0000 7ff6067fc640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6200d36f0 0x7ff620159450 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.689+0000 7ff6067fc640 1 --2- 172.21.15.52:0/643903303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6200d1500 0x7ff6200caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:12.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.689+0000 7ff6067fc640 1 -- 172.21.15.52:0/643903303 >> 172.21.15.52:0/643903303 conn(0x7ff62008eb80 msgr2=0x7ff62006e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:12.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.689+0000 7ff6067fc640 1 -- 172.21.15.52:0/643903303 shutdown_connections 2024-05-11T01:01:12.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:12.689+0000 7ff6067fc640 1 -- 172.21.15.52:0/643903303 wait complete. 2024-05-11T01:01:12.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-05-11T01:01:12.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-05-11T01:01:12.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.6 2024-05-11T01:01:12.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-05-11T01:01:12.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-05-11T01:01:12.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-05-11T01:01:12.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-05-11T01:01:12.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-05-11T01:01:13.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T01:01:13.697 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T01:01:13.699 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T01:01:13.700 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T01:01:13.701 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T01:01:13.702 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T01:01:13.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T01:01:13.708 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T01:01:13.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.869+0000 7fe211d5f640 1 Processor -- start 2024-05-11T01:01:13.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.869+0000 7fe211d5f640 1 -- start start 2024-05-11T01:01:13.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.871+0000 7fe211d5f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe20c0b34f0 0x7fe20c0add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.871+0000 7fe211d5f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c15b410 0x7fe20c0723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.871+0000 7fe211d5f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe20c06e320 con 0x7fe20c072900 2024-05-11T01:01:13.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.871+0000 7fe211d5f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe20c15b870 con 0x7fe20c0b34f0 2024-05-11T01:01:13.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.871+0000 7fe211d5f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe20c0b53b0 con 0x7fe20c15b410 2024-05-11T01:01:13.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.871+0000 7fe20affd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c15b410 0x7fe20c0723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.871+0000 7fe20affd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c15b410 0x7fe20c0723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41160/0 (socket says 172.21.15.52:41160) 2024-05-11T01:01:13.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.871+0000 7fe20affd640 1 -- 172.21.15.52:0/2492980061 learned_addr learned my addr 172.21.15.52:0/2492980061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.872+0000 7fe20b7fe640 1 --2- 172.21.15.52:0/2492980061 >> v2:172.21.15.119:3300/0 conn(0x7fe20c0b34f0 0x7fe20c0add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.872+0000 7fe20affd640 1 -- 172.21.15.52:0/2492980061 >> v2:172.21.15.119:3300/0 conn(0x7fe20c0b34f0 msgr2=0x7fe20c0add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.872+0000 7fe20affd640 1 --2- 172.21.15.52:0/2492980061 >> v2:172.21.15.119:3300/0 conn(0x7fe20c0b34f0 0x7fe20c0add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.872+0000 7fe20affd640 1 -- 172.21.15.52:0/2492980061 >> v1:172.21.15.119:6789/0 conn(0x7fe20c072900 legacy=0x7fe20c0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.872+0000 7fe20affd640 1 -- 172.21.15.52:0/2492980061 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c06e5c0 con 0x7fe20c15b410 2024-05-11T01:01:13.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.873+0000 7fe20affd640 1 --2- 172.21.15.52:0/2492980061 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c15b410 0x7fe20c0723c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe204009e50 tx=0x7fe20402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a0eff87352e1a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.874+0000 7fe20a7fc640 1 -- 172.21.15.52:0/2492980061 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204036070 con 0x7fe20c15b410 2024-05-11T01:01:13.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.874+0000 7fe20a7fc640 1 -- 172.21.15.52:0/2492980061 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe204031600 con 0x7fe20c15b410 2024-05-11T01:01:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.875+0000 7fe211d5f640 1 -- 172.21.15.52:0/2492980061 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c15b410 msgr2=0x7fe20c0723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.875+0000 7fe211d5f640 1 --2- 172.21.15.52:0/2492980061 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c15b410 0x7fe20c0723c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe204009e50 tx=0x7fe20402a6a0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.875+0000 7fe211d5f640 1 -- 172.21.15.52:0/2492980061 shutdown_connections 2024-05-11T01:01:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.875+0000 7fe211d5f640 1 --2- 172.21.15.52:0/2492980061 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c15b410 0x7fe20c0723c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.875+0000 7fe211d5f640 1 --2- 172.21.15.52:0/2492980061 >> v2:172.21.15.119:3300/0 conn(0x7fe20c0b34f0 0x7fe20c0add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.875+0000 7fe211d5f640 1 -- 172.21.15.52:0/2492980061 >> 172.21.15.52:0/2492980061 conn(0x7fe20c0b2a00 msgr2=0x7fe20c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.876+0000 7fe211d5f640 1 -- 172.21.15.52:0/2492980061 shutdown_connections 2024-05-11T01:01:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.876+0000 7fe211d5f640 1 -- 172.21.15.52:0/2492980061 wait complete. 2024-05-11T01:01:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.878+0000 7fe211d5f640 1 Processor -- start 2024-05-11T01:01:13.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.878+0000 7fe211d5f640 1 -- start start 2024-05-11T01:01:13.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.879+0000 7fe211d5f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c072900 0x7fe20c13bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.879+0000 7fe211d5f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe20c0b34f0 0x7fe20c0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.879+0000 7fe211d5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c15b410 0x7fe20c14fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.880+0000 7fe211d5f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c0b4e20 con 0x7fe20c0b34f0 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.880+0000 7fe20affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe20c0b34f0 0x7fe20c0cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.880+0000 7fe20affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe20c0b34f0 0x7fe20c0cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35232/0 (socket says 172.21.15.52:35232) 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.880+0000 7fe20affd640 1 -- 172.21.15.52:0/1372723269 learned_addr learned my addr 172.21.15.52:0/1372723269 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.880+0000 7fe20bfff640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c15b410 0x7fe20c14fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.880+0000 7fe20b7fe640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c072900 0x7fe20c13bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.881+0000 7fe211d5f640 1 -- 172.21.15.52:0/1372723269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c06eb80 con 0x7fe20c15b410 2024-05-11T01:01:13.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.881+0000 7fe211d5f640 1 -- 172.21.15.52:0/1372723269 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe20c0b4fa0 con 0x7fe20c072900 2024-05-11T01:01:13.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.881+0000 7fe20affd640 1 -- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c072900 msgr2=0x7fe20c13bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.881+0000 7fe20affd640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c072900 0x7fe20c13bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.881+0000 7fe20affd640 1 -- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c15b410 msgr2=0x7fe20c14fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.881+0000 7fe20affd640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c15b410 0x7fe20c14fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.881+0000 7fe20affd640 1 -- 172.21.15.52:0/1372723269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c0cfb50 con 0x7fe20c0b34f0 2024-05-11T01:01:13.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.881+0000 7fe20bfff640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c15b410 0x7fe20c14fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:13.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.882+0000 7fe20affd640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe20c0b34f0 0x7fe20c0cf320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe20402a6e0 tx=0x7fe204004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.883+0000 7fe20b7fe640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c072900 0x7fe20c13bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:13.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.883+0000 7fe1f3fff640 1 -- 172.21.15.52:0/1372723269 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204041030 con 0x7fe20c0b34f0 2024-05-11T01:01:13.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.883+0000 7fe211d5f640 1 -- 172.21.15.52:0/1372723269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20c0cfe30 con 0x7fe20c0b34f0 2024-05-11T01:01:13.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.883+0000 7fe211d5f640 1 -- 172.21.15.52:0/1372723269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20c1393d0 con 0x7fe20c0b34f0 2024-05-11T01:01:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.887+0000 7fe1f3fff640 1 -- 172.21.15.52:0/1372723269 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe204037040 con 0x7fe20c0b34f0 2024-05-11T01:01:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.888+0000 7fe1f3fff640 1 -- 172.21.15.52:0/1372723269 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204036070 con 0x7fe20c0b34f0 2024-05-11T01:01:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.888+0000 7fe1f3fff640 1 -- 172.21.15.52:0/1372723269 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe204044070 con 0x7fe20c0b34f0 2024-05-11T01:01:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.889+0000 7fe1f3fff640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe1f8076400 0x7fe1f80788c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.889+0000 7fe1f3fff640 1 -- 172.21.15.52:0/1372723269 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe2040b7f30 con 0x7fe20c0b34f0 2024-05-11T01:01:13.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.891+0000 7fe211d5f640 1 -- 172.21.15.52:0/1372723269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec005290 con 0x7fe20c0b34f0 2024-05-11T01:01:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.896+0000 7fe20b7fe640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe1f8076400 0x7fe1f80788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.901+0000 7fe1f3fff640 1 -- 172.21.15.52:0/1372723269 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe20404b070 con 0x7fe20c0b34f0 2024-05-11T01:01:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.904+0000 7fe20b7fe640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe1f8076400 0x7fe1f80788c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe2000099a0 tx=0x7fe200008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.921+0000 7f5b0af04640 1 Processor -- start 2024-05-11T01:01:13.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.922+0000 7f58c62af640 1 Processor -- start 2024-05-11T01:01:13.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.921+0000 7f5b0af04640 1 -- start start 2024-05-11T01:01:13.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.922+0000 7f5b0af04640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc0df6e0 0x7f5afc054cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.922+0000 7f5b0af04640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5afc050080 0x7f5afc050510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.922+0000 7f5b0af04640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5afc053c20 con 0x7f5afc03ad40 2024-05-11T01:01:13.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.922+0000 7f5b0af04640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5afc053f00 con 0x7f5afc050080 2024-05-11T01:01:13.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.922+0000 7f5b0af04640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5afc0541e0 con 0x7f5afc0df6e0 2024-05-11T01:01:13.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f5b09701640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5afc03ad40 0x7f5afc0533f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:52930/0 (socket says 172.21.15.52:52930) 2024-05-11T01:01:13.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f5b09f02640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc0df6e0 0x7f5afc054cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f5b09701640 1 -- 172.21.15.52:0/3949721088 learned_addr learned my addr 172.21.15.52:0/3949721088 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:13.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f5b0a703640 1 --2- 172.21.15.52:0/3949721088 >> v2:172.21.15.52:3300/0 conn(0x7f5afc050080 0x7f5afc050510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f5b08f00640 1 -- 172.21.15.52:0/3949721088 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 161551872 0 0) 0x7f5afc053c20 con 0x7f5afc03ad40 2024-05-11T01:01:13.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f5b08f00640 1 -- 172.21.15.52:0/3949721088 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5af0003520 con 0x7f5afc03ad40 2024-05-11T01:01:13.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f5b09f02640 1 -- 172.21.15.52:0/3949721088 >> v2:172.21.15.52:3300/0 conn(0x7f5afc050080 msgr2=0x7f5afc050510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f5b09f02640 1 --2- 172.21.15.52:0/3949721088 >> v2:172.21.15.52:3300/0 conn(0x7f5afc050080 0x7f5afc050510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f28d15e5640 1 Processor -- start 2024-05-11T01:01:13.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f28d15e5640 1 -- start start 2024-05-11T01:01:13.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f5b09f02640 1 -- 172.21.15.52:0/3949721088 >> v1:172.21.15.119:6789/0 conn(0x7f5afc03ad40 legacy=0x7f5afc0533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.922+0000 7f58c62af640 1 -- start start 2024-05-11T01:01:13.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c62af640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f58b8006200 0x7f58b8056fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c62af640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f58b80075f0 0x7f58b803c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c62af640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b8057a70 con 0x7f58b80574f0 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c62af640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f58b8057d50 con 0x7f58b8006200 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c62af640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f58b8058030 con 0x7f58b80075f0 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c5aae640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f58b80075f0 0x7f58b803c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c5aae640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f58b80075f0 0x7f58b803c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:46580/0 (socket says 172.21.15.52:46580) 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c5aae640 1 -- 172.21.15.52:0/279098214 learned_addr learned my addr 172.21.15.52:0/279098214 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c52ad640 1 --2- 172.21.15.52:0/279098214 >> v2:172.21.15.52:3300/0 conn(0x7f58b8006200 0x7f58b8056fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c5aae640 1 -- 172.21.15.52:0/279098214 >> v2:172.21.15.52:3300/0 conn(0x7f58b8006200 msgr2=0x7f58b8056fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c5aae640 1 --2- 172.21.15.52:0/279098214 >> v2:172.21.15.52:3300/0 conn(0x7f58b8006200 0x7f58b8056fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c5aae640 1 -- 172.21.15.52:0/279098214 >> v1:172.21.15.52:6789/0 conn(0x7f58b80574f0 legacy=0x7f58b803c010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f58c5aae640 1 -- 172.21.15.52:0/279098214 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b8058310 con 0x7f58b80075f0 2024-05-11T01:01:13.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f5df03e4640 1 Processor -- start 2024-05-11T01:01:13.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.923+0000 7f5df03e4640 1 -- start start 2024-05-11T01:01:13.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f5b09f02640 1 -- 172.21.15.52:0/3949721088 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc0544c0 con 0x7f5afc0df6e0 2024-05-11T01:01:13.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f2957fff640 1 Processor -- start 2024-05-11T01:01:13.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5b25cf6640 1 Processor -- start 2024-05-11T01:01:13.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f5df03e4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5de80702a0 0x7f5de80add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f5df03e4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5de80b56b0 0x7f5de80721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f5df03e4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de80b52b0 con 0x7f5de80b4010 2024-05-11T01:01:13.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f5df03e4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5de806e180 con 0x7f5de80702a0 2024-05-11T01:01:13.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.924+0000 7f5df03e4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5de80b4fd0 con 0x7f5de80b56b0 2024-05-11T01:01:13.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5dee180640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5de80702a0 0x7f5de80add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5dee180640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5de80702a0 0x7f5de80add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35262/0 (socket says 172.21.15.52:35262) 2024-05-11T01:01:13.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5dee180640 1 -- 172.21.15.52:0/3552576724 learned_addr learned my addr 172.21.15.52:0/3552576724 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5ded97f640 1 --2- 172.21.15.52:0/3552576724 >> v2:172.21.15.119:3300/0 conn(0x7f5de80b56b0 0x7f5de80721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5b25cf6640 1 -- start start 2024-05-11T01:01:13.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5b09f02640 1 --2- 172.21.15.52:0/3949721088 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc0df6e0 0x7f5afc054cc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5af800ba40 tx=0x7f5af802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69be3d1e6a45fb07 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f58c5aae640 1 --2- 172.21.15.52:0/279098214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 0x7f58b803c720 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58c0065ab0 tx=0x7f58c00a6720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbbca27a0baed726 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5dee180640 1 -- 172.21.15.52:0/3552576724 >> v2:172.21.15.119:3300/0 conn(0x7f5de80b56b0 msgr2=0x7f5de80721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5dee180640 1 --2- 172.21.15.52:0/3552576724 >> v2:172.21.15.119:3300/0 conn(0x7f5de80b56b0 0x7f5de80721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f28d15e5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f28c4038a40 0x7f28c4054380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f28d15e5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c4038e20 0x7f28c4008350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f28d15e5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c404a260 con 0x7f28c40363b0 2024-05-11T01:01:13.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f28d15e5640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f28c404a4d0 con 0x7f28c4038a40 2024-05-11T01:01:13.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f28d15e5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f28c4053280 con 0x7f28c4038e20 2024-05-11T01:01:13.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5b08f00640 1 -- 172.21.15.52:0/3949721088 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8031600 con 0x7f5afc0df6e0 2024-05-11T01:01:13.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5b08f00640 1 -- 172.21.15.52:0/3949721088 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af80317a0 con 0x7f5afc0df6e0 2024-05-11T01:01:13.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5b08f00640 1 -- 172.21.15.52:0/3949721088 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8031600 con 0x7f5afc0df6e0 2024-05-11T01:01:13.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5dee180640 1 -- 172.21.15.52:0/3552576724 >> v1:172.21.15.119:6789/0 conn(0x7f5de80b4010 legacy=0x7f5de806ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f5dee180640 1 -- 172.21.15.52:0/3552576724 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de806e420 con 0x7f5de80702a0 2024-05-11T01:01:13.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f28d0de4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c4038e20 0x7f28c4008350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f28cb7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f28c40363b0 0x7f28c4007c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:52960/0 (socket says 172.21.15.52:52960) 2024-05-11T01:01:13.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f28cb7fe640 1 -- 172.21.15.52:0/382239125 learned_addr learned my addr 172.21.15.52:0/382239125 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:13.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f58b7fff640 1 -- 172.21.15.52:0/279098214 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c00b2070 con 0x7f58b80075f0 2024-05-11T01:01:13.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4091445602 0 0) 0x7f28c404a260 con 0x7f28c40363b0 2024-05-11T01:01:13.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28bc003520 con 0x7f28c40363b0 2024-05-11T01:01:13.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f5b25cf6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5b180e3260 0x7f5b1804f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f58b7fff640 1 -- 172.21.15.52:0/279098214 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c00ab750 con 0x7f58b80075f0 2024-05-11T01:01:13.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.925+0000 7f2957fff640 1 -- start start 2024-05-11T01:01:13.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f2957fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29500df410 0x7f2950053250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f2957fff640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2950006ae0 con 0x7f295004fa80 2024-05-11T01:01:13.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f2957fff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2950006dc0 con 0x7f29500392c0 2024-05-11T01:01:13.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f2957fff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f2950006890 con 0x7f29500df410 2024-05-11T01:01:13.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f5b25cf6640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b18053380 0x7f5b18037bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f5b25cf6640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b180e3640 con 0x7f5b1804fdb0 2024-05-11T01:01:13.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f5b25cf6640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5b18053930 con 0x7f5b180e3260 2024-05-11T01:01:13.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f5b25cf6640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5b18053c10 con 0x7f5b18053380 2024-05-11T01:01:13.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b24cf4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5b180e3260 0x7f5b1804f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f29567fc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29500df410 0x7f2950053250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b24cf4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5b180e3260 0x7f5b1804f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35290/0 (socket says 172.21.15.52:35290) 2024-05-11T01:01:13.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b24cf4640 1 -- 172.21.15.52:0/185597387 learned_addr learned my addr 172.21.15.52:0/185597387 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b254f5640 1 --2- 172.21.15.52:0/185597387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b18053380 0x7f5b18037bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f29567fc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29500df410 0x7f2950053250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41200/0 (socket says 172.21.15.52:41200) 2024-05-11T01:01:13.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f58c62af640 1 -- 172.21.15.52:0/279098214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 msgr2=0x7f58b803c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f58c62af640 1 --2- 172.21.15.52:0/279098214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 0x7f58b803c720 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58c0065ab0 tx=0x7f58c00a6720 comp rx=0 tx=0).stop 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2445024251 0 0) 0x7f28bc003520 con 0x7f28c40363b0 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c4038e20 msgr2=0x7f28c4008350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c4038e20 0x7f28c4008350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 >> v2:172.21.15.52:3300/0 conn(0x7f28c4038a40 msgr2=0x7f28c4054380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 --2- 172.21.15.52:0/382239125 >> v2:172.21.15.52:3300/0 conn(0x7f28c4038a40 0x7f28c4054380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c4053560 con 0x7f28c40363b0 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f28cc051040 con 0x7f28c40363b0 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 >> v1:172.21.15.119:6789/0 conn(0x7f28c40363b0 legacy=0x7f28c4007c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28bc003a10 0x7f28bc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28bc004800 0x7f28bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28bc0051e0 0x7f28bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28c404a4d0 con 0x7f28bc003a10 2024-05-11T01:01:13.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4053280 con 0x7f28bc0051e0 2024-05-11T01:01:13.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f28bc00c170 con 0x7f28bc004800 2024-05-11T01:01:13.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28cb7fe640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28bc004800 0x7f28bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f5b0af04640 1 -- 172.21.15.52:0/3949721088 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc0df6e0 msgr2=0x7f5afc054cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.926+0000 7f5b0af04640 1 --2- 172.21.15.52:0/3949721088 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc0df6e0 0x7f5afc054cc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5af800ba40 tx=0x7f5af802a5d0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b0af04640 1 -- 172.21.15.52:0/3949721088 shutdown_connections 2024-05-11T01:01:13.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b0af04640 1 --2- 172.21.15.52:0/3949721088 >> v2:172.21.15.52:3300/0 conn(0x7f5afc050080 0x7f5afc050510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b0af04640 1 --2- 172.21.15.52:0/3949721088 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc0df6e0 0x7f5afc054cc0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5af800ba40 tx=0x7f5af802a5d0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b0af04640 1 -- 172.21.15.52:0/3949721088 >> 172.21.15.52:0/3949721088 conn(0x7f5afc055340 msgr2=0x7f5afc055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:13.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b0af04640 1 -- 172.21.15.52:0/3949721088 shutdown_connections 2024-05-11T01:01:13.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b0af04640 1 -- 172.21.15.52:0/3949721088 wait complete. 2024-05-11T01:01:13.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28cbfff640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28bc003a10 0x7f28bc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f28d0de4640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28bc0051e0 0x7f28bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b254f5640 1 -- 172.21.15.52:0/185597387 >> v2:172.21.15.52:3300/0 conn(0x7f5b180e3260 msgr2=0x7f5b1804f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b254f5640 1 --2- 172.21.15.52:0/185597387 >> v2:172.21.15.52:3300/0 conn(0x7f5b180e3260 0x7f5b1804f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b254f5640 1 -- 172.21.15.52:0/185597387 >> v1:172.21.15.52:6789/0 conn(0x7f5b1804fdb0 legacy=0x7f5b18052c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f5b254f5640 1 -- 172.21.15.52:0/185597387 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b18053ef0 con 0x7f5b18053380 2024-05-11T01:01:13.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f58c62af640 1 -- 172.21.15.52:0/279098214 shutdown_connections 2024-05-11T01:01:13.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f58c62af640 1 --2- 172.21.15.52:0/279098214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 0x7f58b803c720 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f58c62af640 1 --2- 172.21.15.52:0/279098214 >> v2:172.21.15.52:3300/0 conn(0x7f58b8006200 0x7f58b8056fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f58c62af640 1 -- 172.21.15.52:0/279098214 >> 172.21.15.52:0/279098214 conn(0x7f58b8058ee0 msgr2=0x7f58b80592f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:13.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f2e58bc8640 1 Processor -- start 2024-05-11T01:01:13.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f2e58bc8640 1 -- start start 2024-05-11T01:01:13.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f28cbfff640 1 -- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28bc004800 msgr2=0x7f28bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f28cbfff640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28bc004800 0x7f28bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f28cbfff640 1 -- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28bc0051e0 msgr2=0x7f28bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f28cbfff640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28bc0051e0 0x7f28bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f28cbfff640 1 -- 172.21.15.52:0/382239125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c4053560 con 0x7f28bc003a10 2024-05-11T01:01:13.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f5b254f5640 1 --2- 172.21.15.52:0/185597387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b18053380 0x7f5b18037bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b2006b4c0 tx=0x7f5b200a66c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f771132d6383a39a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f29567fc640 1 -- 172.21.15.52:0/1017072745 learned_addr learned my addr 172.21.15.52:0/1017072745 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814576784 0 0) 0x7f2950006dc0 con 0x7f29500392c0 2024-05-11T01:01:13.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.927+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2940003520 con 0x7f29500392c0 2024-05-11T01:01:13.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3813859057 0 0) 0x7f2940003520 con 0x7f29500392c0 2024-05-11T01:01:13.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29500df410 msgr2=0x7f2950053250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29500df410 0x7f2950053250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 >> v1:172.21.15.52:6789/0 conn(0x7f295004fa80 legacy=0x7f295004fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29500070a0 con 0x7f29500392c0 2024-05-11T01:01:13.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f29580694f0 con 0x7f29500392c0 2024-05-11T01:01:13.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 >> v1:172.21.15.119:6789/0 conn(0x7f29500392c0 legacy=0x7f2950053790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29400039b0 0x7f2940003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2940004350 0x7f2940004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2940004f30 0x7f294000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2950006890 con 0x7f2940004350 2024-05-11T01:01:13.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f294000bf00 con 0x7f2940004f30 2024-05-11T01:01:13.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f294000c080 con 0x7f29400039b0 2024-05-11T01:01:13.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f5dee180640 1 --2- 172.21.15.52:0/3552576724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5de80702a0 0x7f5de80add50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5de80d1e20 tx=0x7f5dd802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59327d0e9ecd091 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f5ded17e640 1 -- 172.21.15.52:0/3552576724 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8033640 con 0x7f5de80702a0 2024-05-11T01:01:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f5b1f7fe640 1 -- 172.21.15.52:0/185597387 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200b2070 con 0x7f5b18053380 2024-05-11T01:01:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f5b1f7fe640 1 -- 172.21.15.52:0/185597387 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b200ab810 con 0x7f5b18053380 2024-05-11T01:01:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.928+0000 7f5b1f7fe640 1 -- 172.21.15.52:0/185597387 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200abcc0 con 0x7f5b18053380 2024-05-11T01:01:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f5ded17e640 1 -- 172.21.15.52:0/3552576724 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd8038070 con 0x7f5de80702a0 2024-05-11T01:01:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f5ded17e640 1 -- 172.21.15.52:0/3552576724 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8033a00 con 0x7f5de80702a0 2024-05-11T01:01:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f2e58bc8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e4c035060 0x7f2e4c0483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f2956ffd640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29400039b0 0x7f2940003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f29567fc640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2940004350 0x7f2940004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f29567fc640 1 -- 172.21.15.52:0/1017072745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29400039b0 msgr2=0x7f2940003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f29567fc640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29400039b0 0x7f2940003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f29567fc640 1 -- 172.21.15.52:0/1017072745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2940004f30 msgr2=0x7f294000b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T01:01:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f29567fc640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2940004f30 0x7f294000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f29567fc640 1 -- 172.21.15.52:0/1017072745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29500070a0 con 0x7f2940004350 2024-05-11T01:01:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f58c62af640 1 -- 172.21.15.52:0/279098214 shutdown_connections 2024-05-11T01:01:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f2e58bc8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4c048d10 con 0x7f2e4c0e7920 2024-05-11T01:01:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f2e52ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e4c035060 0x7f2e4c0483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f2e52ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e4c035060 0x7f2e4c0483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35318/0 (socket says 172.21.15.52:35318) 2024-05-11T01:01:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f2e52ffd640 1 -- 172.21.15.52:0/1960455577 learned_addr learned my addr 172.21.15.52:0/1960455577 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f58c62af640 1 -- 172.21.15.52:0/279098214 wait complete. 2024-05-11T01:01:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f2e58bc8640 1 -- 172.21.15.52:0/1960455577 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4c0e7d00 con 0x7f2e4c048900 2024-05-11T01:01:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f2e58bc8640 1 -- 172.21.15.52:0/1960455577 --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f2e4c0354f0 con 0x7f2e4c035060 2024-05-11T01:01:13.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b25cf6640 1 -- 172.21.15.52:0/185597387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b18053380 msgr2=0x7f5b18037bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/185597387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b18053380 0x7f5b18037bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b2006b4c0 tx=0x7f5b200a66c0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b25cf6640 1 -- 172.21.15.52:0/185597387 shutdown_connections 2024-05-11T01:01:13.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/185597387 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b18053380 0x7f5b18037bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/185597387 >> v2:172.21.15.52:3300/0 conn(0x7f5b180e3260 0x7f5b1804f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b25cf6640 1 -- 172.21.15.52:0/185597387 >> 172.21.15.52:0/185597387 conn(0x7f5b18009fb0 msgr2=0x7f5b1800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:13.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f28cbfff640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28bc003a10 0x7f28bc003e50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009880 tx=0x7f28bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f2f96b19de300b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b25cf6640 1 -- 172.21.15.52:0/185597387 shutdown_connections 2024-05-11T01:01:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480014885 0 0) 0x7f2e4c0e7d00 con 0x7f2e4c048900 2024-05-11T01:01:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e44003520 con 0x7f2e4c048900 2024-05-11T01:01:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 <== mon.1 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3049343984 0 0) 0x7f2e44003520 con 0x7f2e4c048900 2024-05-11T01:01:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 >> v2:172.21.15.52:3300/0 conn(0x7f2e4c035060 msgr2=0x7f2e4c0483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 --2- 172.21.15.52:0/1960455577 >> v2:172.21.15.52:3300/0 conn(0x7f2e4c035060 0x7f2e4c0483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 >> v1:172.21.15.52:6789/0 conn(0x7f2e4c0e7920 legacy=0x7f2e4c03acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c056080 con 0x7f2e4c048900 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 <== mon.1 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f2e54051040 con 0x7f2e4c048900 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 >> v1:172.21.15.119:6789/0 conn(0x7f2e4c048900 legacy=0x7f2e4c0537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e440039b0 0x7f2e44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44004350 0x7f2e44004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e44005020 0x7f2e4401c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0354f0 con 0x7f2e44004350 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4401cf70 con 0x7f2e440039b0 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4401d170 con 0x7f2e44005020 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f2e52ffd640 1 --2- 172.21.15.52:0/1960455577 >> v2:172.21.15.52:3300/0 conn(0x7f2e4c035060 0x7f2e4c0483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f58c62af640 1 Processor -- start 2024-05-11T01:01:13.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b25cf6640 1 -- 172.21.15.52:0/185597387 wait complete. 2024-05-11T01:01:13.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f5b0af04640 1 Processor -- start 2024-05-11T01:01:13.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.929+0000 7f5b0af04640 1 -- start start 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b0af04640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc03ad40 0x7f5afc038ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b0af04640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5afc050080 0x7f5afc17d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b0af04640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5afc0db060 0x7f5afc0d59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f5b09f02640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc03ad40 0x7f5afc038ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b09f02640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc03ad40 0x7f5afc038ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41234/0 (socket says 172.21.15.52:41234) 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b0af04640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc0063d0 con 0x7f5afc0db060 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b0af04640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc006250 con 0x7f5afc050080 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b0af04640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5afc006550 con 0x7f5afc03ad40 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b0a703640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5afc0db060 0x7f5afc0d59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b09f02640 1 -- 172.21.15.52:0/454178455 learned_addr learned my addr 172.21.15.52:0/454178455 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b09701640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5afc050080 0x7f5afc17d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.930+0000 7f29567fc640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2940004350 0x7f2940004820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29480089c0 tx=0x7f2940005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b5a0792fa2cca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948038070 con 0x7f2940004350 2024-05-11T01:01:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8037070 con 0x7f28bc003a10 2024-05-11T01:01:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b802aab0 con 0x7f28bc003a10 2024-05-11T01:01:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f2e53fff640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e44005020 0x7f2e4401c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f58c62af640 1 -- start start 2024-05-11T01:01:13.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b09f02640 1 -- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5afc050080 msgr2=0x7f5afc17d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b09f02640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5afc050080 0x7f5afc17d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b09f02640 1 -- 172.21.15.52:0/454178455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5afc0db060 msgr2=0x7f5afc0d59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f2955ffb640 1 -- 172.21.15.52:0/1017072745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2948034700 con 0x7f2940004350 2024-05-11T01:01:13.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b09f02640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5afc0db060 0x7f5afc0d59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b09f02640 1 -- 172.21.15.52:0/454178455 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc17daa0 con 0x7f5afc03ad40 2024-05-11T01:01:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f5b09f02640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc03ad40 0x7f5afc038ca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5af8000c00 tx=0x7f5af80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f5af67fc640 1 -- 172.21.15.52:0/454178455 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8040070 con 0x7f5afc03ad40 2024-05-11T01:01:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f5af67fc640 1 -- 172.21.15.52:0/454178455 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af80309e0 con 0x7f5afc03ad40 2024-05-11T01:01:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f5af67fc640 1 -- 172.21.15.52:0/454178455 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8031600 con 0x7f5afc03ad40 2024-05-11T01:01:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5afc17dd80 con 0x7f5afc03ad40 2024-05-11T01:01:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5afc0d51a0 con 0x7f5afc03ad40 2024-05-11T01:01:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f58c62af640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58b8006200 0x7f58b80e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f5b25cf6640 1 Processor -- start 2024-05-11T01:01:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f2e537fe640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e440039b0 0x7f2e44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f2e53fff640 1 -- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e440039b0 msgr2=0x7f2e44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f2e53fff640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e440039b0 0x7f2e44003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f2e53fff640 1 -- 172.21.15.52:0/1960455577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44004350 msgr2=0x7f2e44004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f58c62af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 0x7f58b816c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f28d15e5640 1 -- 172.21.15.52:0/382239125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28bc003a10 msgr2=0x7f28bc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f2957fff640 1 -- 172.21.15.52:0/1017072745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2940004350 msgr2=0x7f2940004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f2957fff640 1 --2- 172.21.15.52:0/1017072745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2940004350 0x7f2940004820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29480089c0 tx=0x7f2940005830 comp rx=0 tx=0).stop 2024-05-11T01:01:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f5b25cf6640 1 -- start start 2024-05-11T01:01:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f58c62af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b80574f0 0x7f58b80e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f5df03e4640 1 -- 172.21.15.52:0/3552576724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5de80702a0 msgr2=0x7f5de80add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f28caffd640 1 -- 172.21.15.52:0/382239125 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8037070 con 0x7f28bc003a10 2024-05-11T01:01:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f28d15e5640 1 --2- 172.21.15.52:0/382239125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28bc003a10 0x7f28bc003e50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009880 tx=0x7f28bc005830 comp rx=0 tx=0).stop 2024-05-11T01:01:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.932+0000 7f28d0de4640 1 -- 172.21.15.52:0/382239125 reap_dead start 2024-05-11T01:01:13.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f5df03e4640 1 --2- 172.21.15.52:0/3552576724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5de80702a0 0x7f5de80add50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5de80d1e20 tx=0x7f5dd802c8b0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f28d15e5640 1 -- 172.21.15.52:0/382239125 shutdown_connections 2024-05-11T01:01:13.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f28d15e5640 1 -- 172.21.15.52:0/382239125 >> 172.21.15.52:0/382239125 conn(0x7f28c40537d0 msgr2=0x7f28c4053be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:13.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f29577fe640 1 -- 172.21.15.52:0/1017072745 reap_dead start 2024-05-11T01:01:13.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f2957fff640 1 -- 172.21.15.52:0/1017072745 shutdown_connections 2024-05-11T01:01:13.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f28d15e5640 1 -- 172.21.15.52:0/382239125 shutdown_connections 2024-05-11T01:01:13.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f2957fff640 1 -- 172.21.15.52:0/1017072745 >> 172.21.15.52:0/1017072745 conn(0x7f2950009fa0 msgr2=0x7f295000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:13.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f2957fff640 1 -- 172.21.15.52:0/1017072745 shutdown_connections 2024-05-11T01:01:13.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f28d15e5640 1 -- 172.21.15.52:0/382239125 wait complete. 2024-05-11T01:01:13.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f2957fff640 1 -- 172.21.15.52:0/1017072745 wait complete. 2024-05-11T01:01:13.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5afc0057d0 con 0x7f5afc03ad40 2024-05-11T01:01:13.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f5df03e4640 1 -- 172.21.15.52:0/3552576724 shutdown_connections 2024-05-11T01:01:13.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f5df03e4640 1 --2- 172.21.15.52:0/3552576724 >> v2:172.21.15.119:3300/0 conn(0x7f5de80b56b0 0x7f5de80721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f5df03e4640 1 --2- 172.21.15.52:0/3552576724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5de80702a0 0x7f5de80add50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f5df03e4640 1 -- 172.21.15.52:0/3552576724 >> 172.21.15.52:0/3552576724 conn(0x7f5de80b2a00 msgr2=0x7f5de80b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:13.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f58c4aac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 0x7f58b816c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f58c4aac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 0x7f58b816c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:46646/0 (socket says 172.21.15.52:46646) 2024-05-11T01:01:13.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f58c4aac640 1 -- 172.21.15.52:0/2668993595 learned_addr learned my addr 172.21.15.52:0/2668993595 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f58c52ad640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58b8006200 0x7f58b80e18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f58c5aae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b80574f0 0x7f58b80e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58b803edd0 con 0x7f58b80574f0 2024-05-11T01:01:13.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f5df03e4640 1 -- 172.21.15.52:0/3552576724 shutdown_connections 2024-05-11T01:01:13.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f5df03e4640 1 -- 172.21.15.52:0/3552576724 wait complete. 2024-05-11T01:01:13.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f5af67fc640 1 -- 172.21.15.52:0/454178455 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5af8039040 con 0x7f5afc03ad40 2024-05-11T01:01:13.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f5b25cf6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b1804fdb0 0x7f5b1804bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.931+0000 7f2e52ffd640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44004350 0x7f2e44004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f2e53fff640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44004350 0x7f2e44004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f2e53fff640 1 -- 172.21.15.52:0/1960455577 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c056080 con 0x7f2e44005020 2024-05-11T01:01:13.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f2e52ffd640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44004350 0x7f2e44004820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:13.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.933+0000 7f2e53fff640 1 -- 172.21.15.52:0/1960455577 reap_dead start 2024-05-11T01:01:13.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f2e53fff640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e44005020 0x7f2e4401c9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e540521a0 tx=0x7f2e44005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8aa6b393646617 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540b4070 con 0x7f2e44005020 2024-05-11T01:01:13.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e540a7840 con 0x7f2e44005020 2024-05-11T01:01:13.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f2e527fc640 1 -- 172.21.15.52:0/1960455577 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540b4070 con 0x7f2e44005020 2024-05-11T01:01:13.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.934+0000 7f5b24cf4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b1804fdb0 0x7f5b1804bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5b24cf4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b1804fdb0 0x7f5b1804bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35348/0 (socket says 172.21.15.52:35348) 2024-05-11T01:01:13.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5b24cf4640 1 -- 172.21.15.52:0/1293307794 learned_addr learned my addr 172.21.15.52:0/1293307794 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80e3520 con 0x7f58b80075f0 2024-05-11T01:01:13.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f58b8007a40 con 0x7f58b8006200 2024-05-11T01:01:13.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b18053380 0x7f5b180dda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b180e3260 0x7f5b180d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180380f0 con 0x7f5b1804fdb0 2024-05-11T01:01:13.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f2e58bc8640 1 -- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e44005020 msgr2=0x7f2e4401c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5b1ffff640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b18053380 0x7f5b180dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f2e58bc8640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e44005020 0x7f2e4401c9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e540521a0 tx=0x7f2e44005830 comp rx=0 tx=0).stop 2024-05-11T01:01:13.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5b254f5640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b180e3260 0x7f5b180d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5af67fc640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ae80762d0 0x7f5ae8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5af67fc640 1 -- 172.21.15.52:0/454178455 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5af80b8890 con 0x7f5afc03ad40 2024-05-11T01:01:13.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f28d15e5640 1 Processor -- start 2024-05-11T01:01:13.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f2e58bc8640 1 -- 172.21.15.52:0/1960455577 shutdown_connections 2024-05-11T01:01:13.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f2e58bc8640 1 --2- 172.21.15.52:0/1960455577 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e44005020 0x7f2e4401c9d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f2e58bc8640 1 -- 172.21.15.52:0/1960455577 >> 172.21.15.52:0/1960455577 conn(0x7f2e4c01e6c0 msgr2=0x7f2e4c0553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:13.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180544b0 con 0x7f5b18053380 2024-05-11T01:01:13.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f2957fff640 1 Processor -- start 2024-05-11T01:01:13.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f2e58bc8640 1 -- 172.21.15.52:0/1960455577 shutdown_connections 2024-05-11T01:01:13.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f2957fff640 1 -- start start 2024-05-11T01:01:13.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5b18038270 con 0x7f5b180e3260 2024-05-11T01:01:13.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f5df03e4640 1 Processor -- start 2024-05-11T01:01:13.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f5b09701640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ae80762d0 0x7f5ae8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f5df03e4640 1 -- start start 2024-05-11T01:01:13.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f2e58bc8640 1 -- 172.21.15.52:0/1960455577 wait complete. 2024-05-11T01:01:13.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.935+0000 7f28d15e5640 1 -- start start 2024-05-11T01:01:13.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f5b24cf4640 1 -- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b180e3260 msgr2=0x7f5b180d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f5b09701640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ae80762d0 0x7f5ae8078790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0396a0 tx=0x7f5b00001090 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f58c4aac640 1 -- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58b8006200 msgr2=0x7f58b80e18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f58c4aac640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58b8006200 0x7f58b80e18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f58c4aac640 1 -- 172.21.15.52:0/2668993595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b80574f0 msgr2=0x7f58b80e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f58c4aac640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b80574f0 0x7f58b80e12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f58c4aac640 1 -- 172.21.15.52:0/2668993595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b816dc20 con 0x7f58b80075f0 2024-05-11T01:01:13.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f58c52ad640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58b8006200 0x7f58b80e18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:13.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f58c5aae640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b80574f0 0x7f58b80e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:13.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f2957fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29500dae10 0x7f2950035310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f2957fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2950175630 0x7f29500e19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f2957fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2950035850 0x7f29500e1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f5b24cf4640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b180e3260 0x7f5b180d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f2957fff640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2950052cb0 con 0x7f2950035850 2024-05-11T01:01:13.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f2957fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2950052b30 con 0x7f2950175630 2024-05-11T01:01:13.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f2957fff640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f295000aa90 con 0x7f29500dae10 2024-05-11T01:01:13.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f58c4aac640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 0x7f58b816c460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58b000b3e0 tx=0x7f58b000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f5b24cf4640 1 -- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b18053380 msgr2=0x7f5b180dda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f5b24cf4640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b18053380 0x7f5b180dda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f29567fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2950175630 0x7f29500e19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f29577fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2950035850 0x7f29500e1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.936+0000 7f5b24cf4640 1 -- 172.21.15.52:0/1293307794 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b180e5270 con 0x7f5b1804fdb0 2024-05-11T01:01:13.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5df03e4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5de80702a0 0x7f5de81510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5df03e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de80b4010 0x7f5de815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5df03e4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5de80b56b0 0x7f5de815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5dee180640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5de80702a0 0x7f5de81510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5df03e4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8070020 con 0x7f5de80b56b0 2024-05-11T01:01:13.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5df03e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5de806fea0 con 0x7f5de80b4010 2024-05-11T01:01:13.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5df03e4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5de815b410 con 0x7f5de80702a0 2024-05-11T01:01:13.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5dee180640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5de80702a0 0x7f5de81510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41284/0 (socket says 172.21.15.52:41284) 2024-05-11T01:01:13.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5dee180640 1 -- 172.21.15.52:0/4018711514 learned_addr learned my addr 172.21.15.52:0/4018711514 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f2e58bc8640 1 Processor -- start 2024-05-11T01:01:13.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f29567fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2950175630 0x7f29500e19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:46672/0 (socket says 172.21.15.52:46672) 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f29567fc640 1 -- 172.21.15.52:0/868137925 learned_addr learned my addr 172.21.15.52:0/868137925 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28d15e5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c40e11a0 0x7f28c4175530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28d15e5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c40e1580 0x7f28c404d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28d15e5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28c404d810 0x7f28c40dd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28d15e5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4049eb0 con 0x7f28c40e1580 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28d15e5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4049d30 con 0x7f28c404d810 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28d15e5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f28c4008350 con 0x7f28c40e11a0 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28cbfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c40e11a0 0x7f28c4175530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28cbfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c40e11a0 0x7f28c4175530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41280/0 (socket says 172.21.15.52:41280) 2024-05-11T01:01:13.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28cbfff640 1 -- 172.21.15.52:0/3252115839 learned_addr learned my addr 172.21.15.52:0/3252115839 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28cb7fe640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c40e1580 0x7f28c404d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f28d0de4640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28c404d810 0x7f28c40dd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f58b5ffb640 1 -- 172.21.15.52:0/2668993595 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0015020 con 0x7f58b80075f0 2024-05-11T01:01:13.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b8179b10 con 0x7f58b80075f0 2024-05-11T01:01:13.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b816e640 con 0x7f58b80075f0 2024-05-11T01:01:13.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f58b5ffb640 1 -- 172.21.15.52:0/2668993595 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b0011850 con 0x7f58b80075f0 2024-05-11T01:01:13.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f58b5ffb640 1 -- 172.21.15.52:0/2668993595 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0011c40 con 0x7f58b80075f0 2024-05-11T01:01:13.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f5ded97f640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de80b4010 0x7f5de815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f5dee981640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5de80b56b0 0x7f5de815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5b24cf4640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b1804fdb0 0x7f5b1804bb90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400d910 tx=0x7f5b1400ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f29567fc640 1 -- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29500dae10 msgr2=0x7f2950035310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T01:01:13.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f29567fc640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29500dae10 0x7f2950035310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f29567fc640 1 -- 172.21.15.52:0/868137925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2950035850 msgr2=0x7f29500e1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f29567fc640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2950035850 0x7f29500e1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f29567fc640 1 -- 172.21.15.52:0/868137925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2950035cf0 con 0x7f2950175630 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5b254f5640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b180e3260 0x7f5b180d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5b1ffff640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b18053380 0x7f5b180dda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5b1d7fa640 1 -- 172.21.15.52:0/1293307794 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1401c070 con 0x7f5b1804fdb0 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5b1d7fa640 1 -- 172.21.15.52:0/1293307794 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b14016ca0 con 0x7f5b1804fdb0 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5b1d7fa640 1 -- 172.21.15.52:0/1293307794 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b140098a0 con 0x7f5b1804fdb0 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b180e54f0 con 0x7f5b1804fdb0 2024-05-11T01:01:13.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.937+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b180e5ab0 con 0x7f5b1804fdb0 2024-05-11T01:01:13.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f5dee180640 1 -- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de80b4010 msgr2=0x7f5de815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f5dee180640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de80b4010 0x7f5de815b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f5dee180640 1 -- 172.21.15.52:0/4018711514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5de80b56b0 msgr2=0x7f5de815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f5dee180640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5de80b56b0 0x7f5de815d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f5dee180640 1 -- 172.21.15.52:0/4018711514 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de815e070 con 0x7f5de80702a0 2024-05-11T01:01:13.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f29567fc640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2950175630 0x7f29500e19c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2948008ae0 tx=0x7f2948004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f293f7fe640 1 -- 172.21.15.52:0/868137925 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948041030 con 0x7f2950175630 2024-05-11T01:01:13.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f293f7fe640 1 -- 172.21.15.52:0/868137925 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2948032820 con 0x7f2950175630 2024-05-11T01:01:13.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f293f7fe640 1 -- 172.21.15.52:0/868137925 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948038070 con 0x7f2950175630 2024-05-11T01:01:13.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f28cbfff640 1 -- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28c404d810 msgr2=0x7f28c40dd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f28cbfff640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28c404d810 0x7f28c40dd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29500485e0 con 0x7f2950175630 2024-05-11T01:01:13.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.938+0000 7f2e58bc8640 1 -- start start 2024-05-11T01:01:13.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2950048ba0 con 0x7f2950175630 2024-05-11T01:01:13.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f28cbfff640 1 -- 172.21.15.52:0/3252115839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c40e1580 msgr2=0x7f28c404d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f28cbfff640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c40e1580 0x7f28c404d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f28cbfff640 1 -- 172.21.15.52:0/3252115839 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c404f170 con 0x7f28c40e11a0 2024-05-11T01:01:13.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f28cbfff640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c40e11a0 0x7f28c4175530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28b802a4e0 tx=0x7f28b80047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f5dee180640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5de80702a0 0x7f5de81510f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5dd802c8f0 tx=0x7f5dd8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.939+0000 7f2e58bc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44005020 0x7f2e4c0d75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2e58bc8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e4c0db0e0 0x7f2e4c049010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2e537fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44005020 0x7f2e4c0d75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2e537fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44005020 0x7f2e4c0d75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:46704/0 (socket says 172.21.15.52:46704) 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2e537fe640 1 -- 172.21.15.52:0/3380257249 learned_addr learned my addr 172.21.15.52:0/3380257249 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2e58bc8640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e4c053a50 0x7f2e4c053ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2e52ffd640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e4c0db0e0 0x7f2e4c049010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c03a4b0 con 0x7f2e4c0db0e0 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c056360 con 0x7f2e44005020 2024-05-11T01:01:13.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c03a790 con 0x7f2e4c053a50 2024-05-11T01:01:13.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b18005390 con 0x7f5b1804fdb0 2024-05-11T01:01:13.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a0005290 con 0x7f58b80075f0 2024-05-11T01:01:13.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29500089c0 con 0x7f2950175630 2024-05-11T01:01:13.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f58b5ffb640 1 -- 172.21.15.52:0/2668993595 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f58b0018020 con 0x7f58b80075f0 2024-05-11T01:01:13.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f2e53fff640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e4c053a50 0x7f2e4c053ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f2e52ffd640 1 -- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e4c053a50 msgr2=0x7f2e4c053ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f2e52ffd640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e4c053a50 0x7f2e4c053ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f2e52ffd640 1 -- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44005020 msgr2=0x7f2e4c0d75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:13.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f2e52ffd640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44005020 0x7f2e4c0d75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:13.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f2e52ffd640 1 -- 172.21.15.52:0/3380257249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c049550 con 0x7f2e4c0db0e0 2024-05-11T01:01:13.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f58b5ffb640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f58a40763a0 0x7f58a4078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f2e52ffd640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e4c0db0e0 0x7f2e4c049010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e40010a00 tx=0x7f2e40010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f28c8ff9640 1 -- 172.21.15.52:0/3252115839 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8037040 con 0x7f28c40e11a0 2024-05-11T01:01:13.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f28c8ff9640 1 -- 172.21.15.52:0/3252115839 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8003c70 con 0x7f28c40e11a0 2024-05-11T01:01:13.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f28c8ff9640 1 -- 172.21.15.52:0/3252115839 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8009480 con 0x7f28c40e11a0 2024-05-11T01:01:13.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f5ddeffd640 1 -- 172.21.15.52:0/4018711514 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8043070 con 0x7f5de80702a0 2024-05-11T01:01:13.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f5ddeffd640 1 -- 172.21.15.52:0/4018711514 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd803c830 con 0x7f5de80702a0 2024-05-11T01:01:13.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.940+0000 7f5ddeffd640 1 -- 172.21.15.52:0/4018711514 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd803cc20 con 0x7f5de80702a0 2024-05-11T01:01:13.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f58c52ad640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f58a40763a0 0x7f58a4078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de815e350 con 0x7f5de80702a0 2024-05-11T01:01:13.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f58b5ffb640 1 -- 172.21.15.52:0/2668993595 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58b00974b0 con 0x7f58b80075f0 2024-05-11T01:01:13.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.942+0000 7f5b1d7fa640 1 -- 172.21.15.52:0/1293307794 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5b14009a40 con 0x7f5b1804fdb0 2024-05-11T01:01:13.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.942+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de815e910 con 0x7f5de80702a0 2024-05-11T01:01:13.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.942+0000 7f293f7fe640 1 -- 172.21.15.52:0/868137925 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2948044030 con 0x7f2950175630 2024-05-11T01:01:13.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.942+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c404f410 con 0x7f28c40e11a0 2024-05-11T01:01:13.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.942+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c404f9d0 con 0x7f28c40e11a0 2024-05-11T01:01:13.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.941+0000 7f5af67fc640 1 -- 172.21.15.52:0/454178455 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5af8081b00 con 0x7f5afc03ad40 2024-05-11T01:01:13.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.942+0000 7f58c52ad640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f58a40763a0 0x7f58a4078860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f58bc006fd0 tx=0x7f58bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.942+0000 7f293f7fe640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29440763a0 0x7f2944078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.942+0000 7f293f7fe640 1 -- 172.21.15.52:0/868137925 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f29480b8390 con 0x7f2950175630 2024-05-11T01:01:13.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.942+0000 7f2956ffd640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29440763a0 0x7f2944078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.943+0000 7f5b1d7fa640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5b040763a0 0x7f5b04078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.943+0000 7f5b1d7fa640 1 -- 172.21.15.52:0/1293307794 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5b1409f7c0 con 0x7f5b1804fdb0 2024-05-11T01:01:13.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.943+0000 7f5b1ffff640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5b040763a0 0x7f5b04078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.943+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c40057e0 con 0x7f28c40e11a0 2024-05-11T01:01:13.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.943+0000 7f2956ffd640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29440763a0 0x7f2944078860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f294c001790 tx=0x7f294c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.943+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de809eaa0 con 0x7f5de80702a0 2024-05-11T01:01:13.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.944+0000 7f5ddeffd640 1 -- 172.21.15.52:0/4018711514 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5dd8046070 con 0x7f5de80702a0 2024-05-11T01:01:13.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.944+0000 7f5b1ffff640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5b040763a0 0x7f5b04078860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c007fd0 tx=0x7f5b0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:13.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.944+0000 7f28c8ff9640 1 -- 172.21.15.52:0/3252115839 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f28b8032640 con 0x7f28c40e11a0 2024-05-11T01:01:13.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.944+0000 7f5ddeffd640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5dcc076350 0x7f5dcc078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:13.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.945+0000 7f5ddeffd640 1 -- 172.21.15.52:0/4018711514 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5dd80b7cf0 con 0x7f5de80702a0 2024-05-11T01:01:13.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.945+0000 7f5ded97f640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5dcc076350 0x7f5dcc078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:13.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.945+0000 7f5ded97f640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5dcc076350 0x7f5dcc078810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5de80b3ca0 tx=0x7f5de4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:14.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.945+0000 7f28c8ff9640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f28b00763d0 0x7f28b0078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.945+0000 7f28c8ff9640 1 -- 172.21.15.52:0/3252115839 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f28b80b78a0 con 0x7f28c40e11a0 2024-05-11T01:01:14.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.946+0000 7f28cb7fe640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f28b00763d0 0x7f28b0078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.946+0000 7f58b5ffb640 1 -- 172.21.15.52:0/2668993595 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f58b001f020 con 0x7f58b80075f0 2024-05-11T01:01:14.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.946+0000 7f28cb7fe640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f28b00763d0 0x7f28b0078890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06ba10 tx=0x7f28cc08f040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:14.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.948+0000 7f2e3ffff640 1 -- 172.21.15.52:0/3380257249 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4000f040 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.948+0000 7f5b1d7fa640 1 -- 172.21.15.52:0/1293307794 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5b180e5270 con 0x7f5b1804fdb0 2024-05-11T01:01:14.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.948+0000 7f2e3ffff640 1 -- 172.21.15.52:0/3380257249 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4000ec80 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.948+0000 7f2e3ffff640 1 -- 172.21.15.52:0/3380257249 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4001c400 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.948+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c177de0 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.948+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c1783a0 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.949+0000 7f5ddeffd640 1 -- 172.21.15.52:0/4018711514 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5de809eaa0 con 0x7f5de80702a0 2024-05-11T01:01:14.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.949+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4c0057e0 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.950+0000 7f293f7fe640 1 -- 172.21.15.52:0/868137925 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2948012040 con 0x7f2950175630 2024-05-11T01:01:14.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.951+0000 7f2e3ffff640 1 -- 172.21.15.52:0/3380257249 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2e400143f0 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.951+0000 7f2e3ffff640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e380763d0 0x7f2e38078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.952+0000 7f2e3ffff640 1 -- 172.21.15.52:0/3380257249 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e40097d60 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.952+0000 7f2e537fe640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e380763d0 0x7f2e38078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.953+0000 7f2e537fe640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e380763d0 0x7f2e38078890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2e48001690 tx=0x7f2e48009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:14.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.955+0000 7f28c8ff9640 1 -- 172.21.15.52:0/3252115839 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f28b8080b90 con 0x7f28c40e11a0 2024-05-11T01:01:14.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:13.958+0000 7f2e3ffff640 1 -- 172.21.15.52:0/3380257249 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2e40018030 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.345+0000 7fe211d5f640 1 -- 172.21.15.52:0/1372723269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe1ec005580 con 0x7fe20c0b34f0 2024-05-11T01:01:14.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.345+0000 7fe1f3fff640 1 -- 172.21.15.52:0/1372723269 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe204081900 con 0x7fe20c0b34f0 2024-05-11T01:01:14.346 INFO:teuthology.orchestra.run.smithi052.stdout:55834574855 2024-05-11T01:01:14.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.349+0000 7fe1f1ffb640 1 -- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe1f8076400 msgr2=0x7fe1f80788c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.349+0000 7fe1f1ffb640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe1f8076400 0x7fe1f80788c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe2000099a0 tx=0x7fe200008040 comp rx=0 tx=0).stop 2024-05-11T01:01:14.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.349+0000 7fe1f1ffb640 1 -- 172.21.15.52:0/1372723269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe20c0b34f0 msgr2=0x7fe20c0cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.349+0000 7fe1f1ffb640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe20c0b34f0 0x7fe20c0cf320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe20402a6e0 tx=0x7fe204004a40 comp rx=0 tx=0).stop 2024-05-11T01:01:14.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.349+0000 7fe1f1ffb640 1 -- 172.21.15.52:0/1372723269 shutdown_connections 2024-05-11T01:01:14.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.349+0000 7fe1f1ffb640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe1f8076400 0x7fe1f80788c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.349+0000 7fe1f1ffb640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c15b410 0x7fe20c14fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.350+0000 7fe1f1ffb640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe20c0b34f0 0x7fe20c0cf320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.350+0000 7fe1f1ffb640 1 --2- 172.21.15.52:0/1372723269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe20c072900 0x7fe20c13bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.350+0000 7fe1f1ffb640 1 -- 172.21.15.52:0/1372723269 >> 172.21.15.52:0/1372723269 conn(0x7fe20c0b2a00 msgr2=0x7fe20c0b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.350+0000 7fe1f1ffb640 1 -- 172.21.15.52:0/1372723269 shutdown_connections 2024-05-11T01:01:14.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.350+0000 7fe1f1ffb640 1 -- 172.21.15.52:0/1372723269 wait complete. 2024-05-11T01:01:14.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-05-11T01:01:14.369 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:14.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.378+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5b18048390 con 0x7f5b1804fdb0 2024-05-11T01:01:14.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.378+0000 7f5b1d7fa640 1 -- 172.21.15.52:0/1293307794 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b14069100 con 0x7f5b1804fdb0 2024-05-11T01:01:14.379 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:14.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5b040763a0 msgr2=0x7f5b04078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5b040763a0 0x7f5b04078860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c007fd0 tx=0x7f5b0c009040 comp rx=0 tx=0).stop 2024-05-11T01:01:14.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b1804fdb0 msgr2=0x7f5b1804bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b1804fdb0 0x7f5b1804bb90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400d910 tx=0x7f5b1400ddd0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 shutdown_connections 2024-05-11T01:01:14.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5b040763a0 0x7f5b04078860 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5b180e3260 0x7f5b180d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b18053380 0x7f5b180dda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 --2- 172.21.15.52:0/1293307794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b1804fdb0 0x7f5b1804bb90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 >> 172.21.15.52:0/1293307794 conn(0x7f5b18009fb0 msgr2=0x7f5b18039c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 shutdown_connections 2024-05-11T01:01:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.380+0000 7f5b25cf6640 1 -- 172.21.15.52:0/1293307794 wait complete. 2024-05-11T01:01:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.391+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5afc03a680 con 0x7f5afc03ad40 2024-05-11T01:01:14.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.392+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f58a0005580 con 0x7f58b80075f0 2024-05-11T01:01:14.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.392+0000 7f5af67fc640 1 -- 172.21.15.52:0/454178455 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5af8085e00 con 0x7f5afc03ad40 2024-05-11T01:01:14.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-05-11T01:01:14.394 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:14.394 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:14.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.392+0000 7f58b5ffb640 1 -- 172.21.15.52:0/2668993595 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58b0067e20 con 0x7f58b80075f0 2024-05-11T01:01:14.395 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ae80762d0 msgr2=0x7f5ae8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ae80762d0 0x7f5ae8078790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0396a0 tx=0x7f5b00001090 comp rx=0 tx=0).stop 2024-05-11T01:01:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc03ad40 msgr2=0x7f5afc038ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc03ad40 0x7f5afc038ca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5af8000c00 tx=0x7f5af80046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 shutdown_connections 2024-05-11T01:01:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ae80762d0 0x7f5ae8078790 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0396a0 tx=0x7f5b00001090 comp rx=0 tx=0).stop 2024-05-11T01:01:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5afc0db060 0x7f5afc0d59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5afc050080 0x7f5afc17d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 --2- 172.21.15.52:0/454178455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5afc03ad40 0x7f5afc038ca0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f58a40763a0 msgr2=0x7f58a4078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f58c62af640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f58a40763a0 0x7f58a4078860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f58bc006fd0 tx=0x7f58bc008040 comp rx=0 tx=0).stop 2024-05-11T01:01:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 msgr2=0x7f58b816c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f58c62af640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 0x7f58b816c460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58b000b3e0 tx=0x7f58b000b8a0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 >> 172.21.15.52:0/454178455 conn(0x7f5afc055340 msgr2=0x7f5afc0de890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 shutdown_connections 2024-05-11T01:01:14.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.396+0000 7f58c62af640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f58a40763a0 0x7f58a4078860 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.396+0000 7f58c62af640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58b80574f0 0x7f58b80e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.396+0000 7f58c62af640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58b80075f0 0x7f58b816c460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.396+0000 7f58c62af640 1 --2- 172.21.15.52:0/2668993595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f58b8006200 0x7f58b80e18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.396+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 >> 172.21.15.52:0/2668993595 conn(0x7f58b8058ee0 msgr2=0x7f58b8050c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 shutdown_connections 2024-05-11T01:01:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.395+0000 7f5b0af04640 1 -- 172.21.15.52:0/454178455 wait complete. 2024-05-11T01:01:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.396+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 shutdown_connections 2024-05-11T01:01:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.396+0000 7f58c62af640 1 -- 172.21.15.52:0/2668993595 wait complete. 2024-05-11T01:01:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.399+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5de8069780 con 0x7f5de80702a0 2024-05-11T01:01:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.399+0000 7f5ddeffd640 1 -- 172.21.15.52:0/4018711514 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5dd8081050 con 0x7f5de80702a0 2024-05-11T01:01:14.407 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.401+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2950053200 con 0x7f2950175630 2024-05-11T01:01:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.401+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5dcc076350 msgr2=0x7f5dcc078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.401+0000 7f5df03e4640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5dcc076350 0x7f5dcc078810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5de80b3ca0 tx=0x7f5de4009190 comp rx=0 tx=0).stop 2024-05-11T01:01:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.401+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5de80702a0 msgr2=0x7f5de81510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.401+0000 7f5df03e4640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5de80702a0 0x7f5de81510f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5dd802c8f0 tx=0x7f5dd8004730 comp rx=0 tx=0).stop 2024-05-11T01:01:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.402+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 shutdown_connections 2024-05-11T01:01:14.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.402+0000 7f5df03e4640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5dcc076350 0x7f5dcc078810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.402+0000 7f5df03e4640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5de80b56b0 0x7f5de815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.402+0000 7f5df03e4640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de80b4010 0x7f5de815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.402+0000 7f5df03e4640 1 --2- 172.21.15.52:0/4018711514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5de80702a0 0x7f5de81510f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.402+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 >> 172.21.15.52:0/4018711514 conn(0x7f5de80b2a00 msgr2=0x7f5de80731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.402+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 shutdown_connections 2024-05-11T01:01:14.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.402+0000 7f5df03e4640 1 -- 172.21.15.52:0/4018711514 wait complete. 2024-05-11T01:01:14.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.402+0000 7f293f7fe640 1 -- 172.21.15.52:0/868137925 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2948081710 con 0x7f2950175630 2024-05-11T01:01:14.412 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:14.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.405+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29440763a0 msgr2=0x7f2944078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.405+0000 7f2957fff640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29440763a0 0x7f2944078860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f294c001790 tx=0x7f294c009040 comp rx=0 tx=0).stop 2024-05-11T01:01:14.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.405+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2950175630 msgr2=0x7f29500e19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.405+0000 7f2957fff640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2950175630 0x7f29500e19c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2948008ae0 tx=0x7f2948004730 comp rx=0 tx=0).stop 2024-05-11T01:01:14.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.406+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f28c403ccd0 con 0x7f28c40e11a0 2024-05-11T01:01:14.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.406+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 shutdown_connections 2024-05-11T01:01:14.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.406+0000 7f2957fff640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29440763a0 0x7f2944078860 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.406+0000 7f2957fff640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2950035850 0x7f29500e1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.406+0000 7f2957fff640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2950175630 0x7f29500e19c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.407+0000 7f2957fff640 1 --2- 172.21.15.52:0/868137925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29500dae10 0x7f2950035310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.407+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 >> 172.21.15.52:0/868137925 conn(0x7f2950009fa0 msgr2=0x7f29500de080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.407+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 shutdown_connections 2024-05-11T01:01:14.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.407+0000 7f2957fff640 1 -- 172.21.15.52:0/868137925 wait complete. 2024-05-11T01:01:14.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.407+0000 7f28c8ff9640 1 -- 172.21.15.52:0/3252115839 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28b80891d0 con 0x7f28c40e11a0 2024-05-11T01:01:14.418 INFO:teuthology.orchestra.run.smithi052.stdout:55834574854 2024-05-11T01:01:14.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f28b00763d0 msgr2=0x7f28b0078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f28b00763d0 0x7f28b0078890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06ba10 tx=0x7f28cc08f040 comp rx=0 tx=0).stop 2024-05-11T01:01:14.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c40e11a0 msgr2=0x7f28c4175530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c40e11a0 0x7f28c4175530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28b802a4e0 tx=0x7f28b80047d0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 shutdown_connections 2024-05-11T01:01:14.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2e4c005e50 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f28b00763d0 0x7f28b0078890 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28c404d810 0x7f28c40dd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c40e1580 0x7f28c404d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 --2- 172.21.15.52:0/3252115839 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28c40e11a0 0x7f28c4175530 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.410+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 >> 172.21.15.52:0/3252115839 conn(0x7f28c40537d0 msgr2=0x7f28c40df590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.411+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 shutdown_connections 2024-05-11T01:01:14.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.411+0000 7f28d15e5640 1 -- 172.21.15.52:0/3252115839 wait complete. 2024-05-11T01:01:14.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.411+0000 7f2e3ffff640 1 -- 172.21.15.52:0/3380257249 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e400616a0 con 0x7f2e4c0db0e0 2024-05-11T01:01:14.423 INFO:teuthology.orchestra.run.smithi052.stdout:60129542152 2024-05-11T01:01:14.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-05-11T01:01:14.425 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:14.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-05-11T01:01:14.426 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e380763d0 msgr2=0x7f2e38078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e380763d0 0x7f2e38078890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2e48001690 tx=0x7f2e48009040 comp rx=0 tx=0).stop 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e4c0db0e0 msgr2=0x7f2e4c049010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e4c0db0e0 0x7f2e4c049010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e40010a00 tx=0x7f2e40010ec0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 shutdown_connections 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e380763d0 0x7f2e38078890 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e4c053a50 0x7f2e4c053ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e4c0db0e0 0x7f2e4c049010 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 --2- 172.21.15.52:0/3380257249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44005020 0x7f2e4c0d75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 >> 172.21.15.52:0/3380257249 conn(0x7f2e4c01e6c0 msgr2=0x7f2e4c033fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 shutdown_connections 2024-05-11T01:01:14.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.414+0000 7f2e58bc8640 1 -- 172.21.15.52:0/3380257249 wait complete. 2024-05-11T01:01:14.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-05-11T01:01:14.429 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:14.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-05-11T01:01:14.429 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:14.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-11T01:01:14.430 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:14.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.6 2024-05-11T01:01:14.431 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:14.431 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T01:01:14.431 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:01:14.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.571+0000 7fefc77e4640 1 Processor -- start 2024-05-11T01:01:14.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.571+0000 7fefc77e4640 1 -- start start 2024-05-11T01:01:14.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.572+0000 7fefc77e4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc015f5c0 0x7fefc00cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.572+0000 7fefc77e4640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc00d0890 con 0x7fefc0163680 2024-05-11T01:01:14.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.572+0000 7fefc77e4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc00d0b00 con 0x7fefc00d15e0 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.572+0000 7fefc77e4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fefc00d0de0 con 0x7fefc015f5c0 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.572+0000 7fefc5d81640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fefc0163680 0x7fefc00cb850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58148/0 (socket says 172.21.15.52:58148) 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.572+0000 7fefc5d81640 1 -- 172.21.15.52:0/3616553582 learned_addr learned my addr 172.21.15.52:0/3616553582 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.572+0000 7fefc5580640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc015f5c0 0x7fefc00cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.572+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1456254318 0 0) 0x7fefc00d0890 con 0x7fefc0163680 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefac003520 con 0x7fefc0163680 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1168019114 0 0) 0x7fefac003520 con 0x7fefc0163680 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc015f5c0 msgr2=0x7fefc00cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc015f5c0 0x7fefc00cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 >> v1:172.21.15.119:6789/0 conn(0x7fefc00d15e0 legacy=0x7fefc0162f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc00d10c0 con 0x7fefc0163680 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fefb8001a70 con 0x7fefc0163680 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 >> v1:172.21.15.52:6789/0 conn(0x7fefc0163680 legacy=0x7fefc00cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc015f5c0 0x7fefac003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefac004d40 0x7fefac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fefc00d0de0 con 0x7fefac004000 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefac00bc00 con 0x7fefac004d40 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fefac00be00 con 0x7fefc015f5c0 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefc4d7f640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefc5580640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc015f5c0 0x7fefac003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefc5d81640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefac004d40 0x7fefac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefc4d7f640 1 -- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc015f5c0 msgr2=0x7fefac003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefc4d7f640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc015f5c0 0x7fefac003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefc4d7f640 1 -- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefac004d40 msgr2=0x7fefac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefc4d7f640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefac004d40 0x7fefac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.573+0000 7fefc4d7f640 1 -- 172.21.15.52:0/3616553582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac003520 con 0x7fefac004000 2024-05-11T01:01:14.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.574+0000 7fefc4d7f640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefac004800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefb0001790 tx=0x7fefac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793c5d20ef1c9324 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.574+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00013b0 con 0x7fefac004000 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.574+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb00052d0 con 0x7fefac004000 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.574+0000 7fefb7fff640 1 -- 172.21.15.52:0/3616553582 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb002aae0 con 0x7fefac004000 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.575+0000 7fefc77e4640 1 -- 172.21.15.52:0/3616553582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 msgr2=0x7fefac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.575+0000 7fefc77e4640 1 --2- 172.21.15.52:0/3616553582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefac004800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefb0001790 tx=0x7fefac005830 comp rx=0 tx=0).stop 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.575+0000 7fefc5d81640 1 -- 172.21.15.52:0/3616553582 reap_dead start 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.575+0000 7fefc77e4640 1 -- 172.21.15.52:0/3616553582 shutdown_connections 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.575+0000 7fefc77e4640 1 -- 172.21.15.52:0/3616553582 >> 172.21.15.52:0/3616553582 conn(0x7fefc0072aa0 msgr2=0x7fefc0073950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.575+0000 7fefc77e4640 1 -- 172.21.15.52:0/3616553582 shutdown_connections 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.575+0000 7fefc77e4640 1 -- 172.21.15.52:0/3616553582 wait complete. 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.576+0000 7fefc77e4640 1 Processor -- start 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.576+0000 7fefc77e4640 1 -- start start 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc77e4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefc00b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc77e4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc01619a0 0x7fefc00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc77e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc01f9080 0x7fefc01f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc77e4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fefc0163db0 con 0x7fefac004000 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc77e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefc0163c30 con 0x7fefc01f9080 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc77e4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fefc00ce590 con 0x7fefc01619a0 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5580640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefc00b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5580640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefc00b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35386/0 (socket says 172.21.15.52:35386) 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5580640 1 -- 172.21.15.52:0/1230695191 learned_addr learned my addr 172.21.15.52:0/1230695191 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc4d7f640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc01619a0 0x7fefc00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5d81640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc01f9080 0x7fefc01f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5580640 1 -- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc01619a0 msgr2=0x7fefc00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5580640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc01619a0 0x7fefc00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5580640 1 -- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc01f9080 msgr2=0x7fefc01f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5580640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc01f9080 0x7fefc01f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5580640 1 -- 172.21.15.52:0/1230695191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc0150f00 con 0x7fefac004000 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5d81640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc01f9080 0x7fefc01f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:14.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefc5580640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefc00b6980 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0119a0 tx=0x7fefbc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:14.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.577+0000 7fefb5ffb640 1 -- 172.21.15.52:0/1230695191 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00faf0 con 0x7fefac004000 2024-05-11T01:01:14.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.578+0000 7fefb5ffb640 1 -- 172.21.15.52:0/1230695191 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc00fc90 con 0x7fefac004000 2024-05-11T01:01:14.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.578+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc0150940 con 0x7fefac004000 2024-05-11T01:01:14.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.578+0000 7fefb5ffb640 1 -- 172.21.15.52:0/1230695191 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc014440 con 0x7fefac004000 2024-05-11T01:01:14.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.578+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc00b60f0 con 0x7fefac004000 2024-05-11T01:01:14.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.579+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc0159e40 con 0x7fefac004000 2024-05-11T01:01:14.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.579+0000 7fefb5ffb640 1 -- 172.21.15.52:0/1230695191 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fefbc012040 con 0x7fefac004000 2024-05-11T01:01:14.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.580+0000 7fefb5ffb640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fefa4076300 0x7fefa40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.580+0000 7fefb5ffb640 1 -- 172.21.15.52:0/1230695191 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fefbc097ca0 con 0x7fefac004000 2024-05-11T01:01:14.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.580+0000 7fefc4d7f640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fefa4076300 0x7fefa40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.581+0000 7fefc4d7f640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fefa4076300 0x7fefa40787c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fefb0032000 tx=0x7fefb002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:14.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.584+0000 7fefb5ffb640 1 -- 172.21.15.52:0/1230695191 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fefc0159e40 con 0x7fefac004000 2024-05-11T01:01:14.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.773+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefc0069780 con 0x7fefa4076300 2024-05-11T01:01:14.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.780+0000 7fefb5ffb640 1 -- 172.21.15.52:0/1230695191 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7fefc0069780 con 0x7fefa4076300 2024-05-11T01:01:14.783 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:14.784 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fefa4076300 msgr2=0x7fefa40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fefa4076300 0x7fefa40787c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fefb0032000 tx=0x7fefb002e000 comp rx=0 tx=0).stop 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 msgr2=0x7fefc00b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefc00b6980 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0119a0 tx=0x7fefbc011e60 comp rx=0 tx=0).stop 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 shutdown_connections 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fefa4076300 0x7fefa40787c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc01f9080 0x7fefc01f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc01619a0 0x7fefc00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 --2- 172.21.15.52:0/1230695191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefac004000 0x7fefc00b6980 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 >> 172.21.15.52:0/1230695191 conn(0x7fefc0072aa0 msgr2=0x7fefc006f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 shutdown_connections 2024-05-11T01:01:14.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.782+0000 7fefc77e4640 1 -- 172.21.15.52:0/1230695191 wait complete. 2024-05-11T01:01:14.795 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-05-11T01:01:13.513017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235632,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739088,"statfs":{"total":773094113280,"available":772852826112,"internally_reserved":0,"allocated":2891776,"data_stored":1750788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.837260"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794175+0000","last_change":"2024-05-11T01:01:01.690368+0000","last_active":"2024-05-11T01:01:02.794175+0000","last_peered":"2024-05-11T01:01:02.794175+0000","last_clean":"2024-05-11T01:01:02.794175+0000","last_became_active":"2024-05-11T01:01:01.689860+0000","last_became_peered":"2024-05-11T01:01:01.689860+0000","last_unstale":"2024-05-11T01:01:02.794175+0000","last_undegraded":"2024-05-11T01:01:02.794175+0000","last_fullsized":"2024-05-11T01:01:02.794175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:22:27.780116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703164+0000","last_change":"2024-05-11T01:01:01.692121+0000","last_active":"2024-05-11T01:01:02.703164+0000","last_peered":"2024-05-11T01:01:02.703164+0000","last_clean":"2024-05-11T01:01:02.703164+0000","last_became_active":"2024-05-11T01:01:01.691843+0000","last_became_peered":"2024-05-11T01:01:01.691843+0000","last_unstale":"2024-05-11T01:01:02.703164+0000","last_undegraded":"2024-05-11T01:01:02.703164+0000","last_fullsized":"2024-05-11T01:01:02.703164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:04:19.015364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676930+0000","last_change":"2024-05-11T01:01:01.681630+0000","last_active":"2024-05-11T01:01:02.676930+0000","last_peered":"2024-05-11T01:01:02.676930+0000","last_clean":"2024-05-11T01:01:02.676930+0000","last_became_active":"2024-05-11T01:01:01.681342+0000","last_became_peered":"2024-05-11T01:01:01.681342+0000","last_unstale":"2024-05-11T01:01:02.676930+0000","last_undegraded":"2024-05-11T01:01:02.676930+0000","last_fullsized":"2024-05-11T01:01:02.676930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:02:55.828068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678747+0000","last_change":"2024-05-11T01:01:01.687729+0000","last_active":"2024-05-11T01:01:02.678747+0000","last_peered":"2024-05-11T01:01:02.678747+0000","last_clean":"2024-05-11T01:01:02.678747+0000","last_became_active":"2024-05-11T01:01:01.687234+0000","last_became_peered":"2024-05-11T01:01:01.687234+0000","last_unstale":"2024-05-11T01:01:02.678747+0000","last_undegraded":"2024-05-11T01:01:02.678747+0000","last_fullsized":"2024-05-11T01:01:02.678747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:10.357158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793604+0000","last_change":"2024-05-11T01:01:01.687598+0000","last_active":"2024-05-11T01:01:02.793604+0000","last_peered":"2024-05-11T01:01:02.793604+0000","last_clean":"2024-05-11T01:01:02.793604+0000","last_became_active":"2024-05-11T01:01:01.687326+0000","last_became_peered":"2024-05-11T01:01:01.687326+0000","last_unstale":"2024-05-11T01:01:02.793604+0000","last_undegraded":"2024-05-11T01:01:02.793604+0000","last_fullsized":"2024-05-11T01:01:02.793604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677883+0000","last_change":"2024-05-11T01:01:01.688617+0000","last_active":"2024-05-11T01:01:02.677883+0000","last_peered":"2024-05-11T01:01:02.677883+0000","last_clean":"2024-05-11T01:01:02.677883+0000","last_became_active":"2024-05-11T01:01:01.687985+0000","last_became_peered":"2024-05-11T01:01:01.687985+0000","last_unstale":"2024-05-11T01:01:02.677883+0000","last_undegraded":"2024-05-11T01:01:02.677883+0000","last_fullsized":"2024-05-11T01:01:02.677883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:24.445230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793490+0000","last_change":"2024-05-11T01:01:01.692158+0000","last_active":"2024-05-11T01:01:02.793490+0000","last_peered":"2024-05-11T01:01:02.793490+0000","last_clean":"2024-05-11T01:01:02.793490+0000","last_became_active":"2024-05-11T01:01:01.691929+0000","last_became_peered":"2024-05-11T01:01:01.691929+0000","last_unstale":"2024-05-11T01:01:02.793490+0000","last_undegraded":"2024-05-11T01:01:02.793490+0000","last_fullsized":"2024-05-11T01:01:02.793490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:24.858093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912954+0000","last_change":"2024-05-11T01:01:01.694128+0000","last_active":"2024-05-11T01:01:02.912954+0000","last_peered":"2024-05-11T01:01:02.912954+0000","last_clean":"2024-05-11T01:01:02.912954+0000","last_became_active":"2024-05-11T01:01:01.693513+0000","last_became_peered":"2024-05-11T01:01:01.693513+0000","last_unstale":"2024-05-11T01:01:02.912954+0000","last_undegraded":"2024-05-11T01:01:02.912954+0000","last_fullsized":"2024-05-11T01:01:02.912954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:02:55.828068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794149+0000","last_change":"2024-05-11T01:01:01.705504+0000","last_active":"2024-05-11T01:01:02.794149+0000","last_peered":"2024-05-11T01:01:02.794149+0000","last_clean":"2024-05-11T01:01:02.794149+0000","last_became_active":"2024-05-11T01:01:01.705258+0000","last_became_peered":"2024-05-11T01:01:01.705258+0000","last_unstale":"2024-05-11T01:01:02.794149+0000","last_undegraded":"2024-05-11T01:01:02.794149+0000","last_fullsized":"2024-05-11T01:01:02.794149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:00:03.997273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705744+0000","last_change":"2024-05-11T01:01:01.696662+0000","last_active":"2024-05-11T01:01:02.705744+0000","last_peered":"2024-05-11T01:01:02.705744+0000","last_clean":"2024-05-11T01:01:02.705744+0000","last_became_active":"2024-05-11T01:01:01.696364+0000","last_became_peered":"2024-05-11T01:01:01.696364+0000","last_unstale":"2024-05-11T01:01:02.705744+0000","last_undegraded":"2024-05-11T01:01:02.705744+0000","last_fullsized":"2024-05-11T01:01:02.705744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:04:19.015364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705571+0000","last_change":"2024-05-11T01:01:01.699790+0000","last_active":"2024-05-11T01:01:02.705571+0000","last_peered":"2024-05-11T01:01:02.705571+0000","last_clean":"2024-05-11T01:01:02.705571+0000","last_became_active":"2024-05-11T01:01:01.699494+0000","last_became_peered":"2024-05-11T01:01:01.699494+0000","last_unstale":"2024-05-11T01:01:02.705571+0000","last_undegraded":"2024-05-11T01:01:02.705571+0000","last_fullsized":"2024-05-11T01:01:02.705571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678303+0000","last_change":"2024-05-11T01:01:01.696604+0000","last_active":"2024-05-11T01:01:02.678303+0000","last_peered":"2024-05-11T01:01:02.678303+0000","last_clean":"2024-05-11T01:01:02.678303+0000","last_became_active":"2024-05-11T01:01:01.695857+0000","last_became_peered":"2024-05-11T01:01:01.695857+0000","last_unstale":"2024-05-11T01:01:02.678303+0000","last_undegraded":"2024-05-11T01:01:02.678303+0000","last_fullsized":"2024-05-11T01:01:02.678303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:06.977149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913350+0000","last_change":"2024-05-11T01:01:01.690950+0000","last_active":"2024-05-11T01:01:02.913350+0000","last_peered":"2024-05-11T01:01:02.913350+0000","last_clean":"2024-05-11T01:01:02.913350+0000","last_became_active":"2024-05-11T01:01:01.690683+0000","last_became_peered":"2024-05-11T01:01:01.690683+0000","last_unstale":"2024-05-11T01:01:02.913350+0000","last_undegraded":"2024-05-11T01:01:02.913350+0000","last_fullsized":"2024-05-11T01:01:02.913350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676647+0000","last_change":"2024-05-11T01:01:01.690525+0000","last_active":"2024-05-11T01:01:02.676647+0000","last_peered":"2024-05-11T01:01:02.676647+0000","last_clean":"2024-05-11T01:01:02.676647+0000","last_became_active":"2024-05-11T01:01:01.690246+0000","last_became_peered":"2024-05-11T01:01:01.690246+0000","last_unstale":"2024-05-11T01:01:02.676647+0000","last_undegraded":"2024-05-11T01:01:02.676647+0000","last_fullsized":"2024-05-11T01:01:02.676647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:59.768543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.702977+0000","last_change":"2024-05-11T01:01:01.696660+0000","last_active":"2024-05-11T01:01:02.702977+0000","last_peered":"2024-05-11T01:01:02.702977+0000","last_clean":"2024-05-11T01:01:02.702977+0000","last_became_active":"2024-05-11T01:01:01.696344+0000","last_became_peered":"2024-05-11T01:01:01.696344+0000","last_unstale":"2024-05-11T01:01:02.702977+0000","last_undegraded":"2024-05-11T01:01:02.702977+0000","last_fullsized":"2024-05-11T01:01:02.702977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677628+0000","last_change":"2024-05-11T01:01:01.691118+0000","last_active":"2024-05-11T01:01:02.677628+0000","last_peered":"2024-05-11T01:01:02.677628+0000","last_clean":"2024-05-11T01:01:02.677628+0000","last_became_active":"2024-05-11T01:01:01.690850+0000","last_became_peered":"2024-05-11T01:01:01.690850+0000","last_unstale":"2024-05-11T01:01:02.677628+0000","last_undegraded":"2024-05-11T01:01:02.677628+0000","last_fullsized":"2024-05-11T01:01:02.677628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678640+0000","last_change":"2024-05-11T01:01:01.691696+0000","last_active":"2024-05-11T01:01:02.678640+0000","last_peered":"2024-05-11T01:01:02.678640+0000","last_clean":"2024-05-11T01:01:02.678640+0000","last_became_active":"2024-05-11T01:01:01.691424+0000","last_became_peered":"2024-05-11T01:01:01.691424+0000","last_unstale":"2024-05-11T01:01:02.678640+0000","last_undegraded":"2024-05-11T01:01:02.678640+0000","last_fullsized":"2024-05-11T01:01:02.678640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:13:49.063574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913015+0000","last_change":"2024-05-11T01:01:01.690778+0000","last_active":"2024-05-11T01:01:02.913015+0000","last_peered":"2024-05-11T01:01:02.913015+0000","last_clean":"2024-05-11T01:01:02.913015+0000","last_became_active":"2024-05-11T01:01:01.690403+0000","last_became_peered":"2024-05-11T01:01:01.690403+0000","last_unstale":"2024-05-11T01:01:02.913015+0000","last_undegraded":"2024-05-11T01:01:02.913015+0000","last_fullsized":"2024-05-11T01:01:02.913015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:59.768543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794101+0000","last_change":"2024-05-11T01:01:01.697511+0000","last_active":"2024-05-11T01:01:02.794101+0000","last_peered":"2024-05-11T01:01:02.794101+0000","last_clean":"2024-05-11T01:01:02.794101+0000","last_became_active":"2024-05-11T01:01:01.697263+0000","last_became_peered":"2024-05-11T01:01:01.697263+0000","last_unstale":"2024-05-11T01:01:02.794101+0000","last_undegraded":"2024-05-11T01:01:02.794101+0000","last_fullsized":"2024-05-11T01:01:02.794101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:10.357158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705536+0000","last_change":"2024-05-11T01:01:01.697194+0000","last_active":"2024-05-11T01:01:02.705536+0000","last_peered":"2024-05-11T01:01:02.705536+0000","last_clean":"2024-05-11T01:01:02.705536+0000","last_became_active":"2024-05-11T01:01:01.696909+0000","last_became_peered":"2024-05-11T01:01:01.696909+0000","last_unstale":"2024-05-11T01:01:02.705536+0000","last_undegraded":"2024-05-11T01:01:02.705536+0000","last_fullsized":"2024-05-11T01:01:02.705536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676985+0000","last_change":"2024-05-11T01:01:01.690623+0000","last_active":"2024-05-11T01:01:02.676985+0000","last_peered":"2024-05-11T01:01:02.676985+0000","last_clean":"2024-05-11T01:01:02.676985+0000","last_became_active":"2024-05-11T01:01:01.689762+0000","last_became_peered":"2024-05-11T01:01:01.689762+0000","last_unstale":"2024-05-11T01:01:02.676985+0000","last_undegraded":"2024-05-11T01:01:02.676985+0000","last_fullsized":"2024-05-11T01:01:02.676985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:10:39.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678044+0000","last_change":"2024-05-11T01:01:01.692409+0000","last_active":"2024-05-11T01:01:02.678044+0000","last_peered":"2024-05-11T01:01:02.678044+0000","last_clean":"2024-05-11T01:01:02.678044+0000","last_became_active":"2024-05-11T01:01:01.692117+0000","last_became_peered":"2024-05-11T01:01:01.692117+0000","last_unstale":"2024-05-11T01:01:02.678044+0000","last_undegraded":"2024-05-11T01:01:02.678044+0000","last_fullsized":"2024-05-11T01:01:02.678044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:36:33.062657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913338+0000","last_change":"2024-05-11T01:01:01.697812+0000","last_active":"2024-05-11T01:01:02.913338+0000","last_peered":"2024-05-11T01:01:02.913338+0000","last_clean":"2024-05-11T01:01:02.913338+0000","last_became_active":"2024-05-11T01:01:01.697509+0000","last_became_peered":"2024-05-11T01:01:01.697509+0000","last_unstale":"2024-05-11T01:01:02.913338+0000","last_undegraded":"2024-05-11T01:01:02.913338+0000","last_fullsized":"2024-05-11T01:01:02.913338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678573+0000","last_change":"2024-05-11T01:01:01.689599+0000","last_active":"2024-05-11T01:01:02.678573+0000","last_peered":"2024-05-11T01:01:02.678573+0000","last_clean":"2024-05-11T01:01:02.678573+0000","last_became_active":"2024-05-11T01:01:01.686994+0000","last_became_peered":"2024-05-11T01:01:01.686994+0000","last_unstale":"2024-05-11T01:01:02.678573+0000","last_undegraded":"2024-05-11T01:01:02.678573+0000","last_fullsized":"2024-05-11T01:01:02.678573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:32.857796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913719+0000","last_change":"2024-05-11T01:01:01.699000+0000","last_active":"2024-05-11T01:01:02.913719+0000","last_peered":"2024-05-11T01:01:02.913719+0000","last_clean":"2024-05-11T01:01:02.913719+0000","last_became_active":"2024-05-11T01:01:01.698640+0000","last_became_peered":"2024-05-11T01:01:01.698640+0000","last_unstale":"2024-05-11T01:01:02.913719+0000","last_undegraded":"2024-05-11T01:01:02.913719+0000","last_fullsized":"2024-05-11T01:01:02.913719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:24.445230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705756+0000","last_change":"2024-05-11T01:01:01.689683+0000","last_active":"2024-05-11T01:01:02.705756+0000","last_peered":"2024-05-11T01:01:02.705756+0000","last_clean":"2024-05-11T01:01:02.705756+0000","last_became_active":"2024-05-11T01:01:01.689306+0000","last_became_peered":"2024-05-11T01:01:01.689306+0000","last_unstale":"2024-05-11T01:01:02.705756+0000","last_undegraded":"2024-05-11T01:01:02.705756+0000","last_fullsized":"2024-05-11T01:01:02.705756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703176+0000","last_change":"2024-05-11T01:01:01.694512+0000","last_active":"2024-05-11T01:01:02.703176+0000","last_peered":"2024-05-11T01:01:02.703176+0000","last_clean":"2024-05-11T01:01:02.703176+0000","last_became_active":"2024-05-11T01:01:01.694214+0000","last_became_peered":"2024-05-11T01:01:01.694214+0000","last_unstale":"2024-05-11T01:01:02.703176+0000","last_undegraded":"2024-05-11T01:01:02.703176+0000","last_fullsized":"2024-05-11T01:01:02.703176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913695+0000","last_change":"2024-05-11T01:01:01.698983+0000","last_active":"2024-05-11T01:01:02.913695+0000","last_peered":"2024-05-11T01:01:02.913695+0000","last_clean":"2024-05-11T01:01:02.913695+0000","last_became_active":"2024-05-11T01:01:01.698682+0000","last_became_peered":"2024-05-11T01:01:01.698682+0000","last_unstale":"2024-05-11T01:01:02.913695+0000","last_undegraded":"2024-05-11T01:01:02.913695+0000","last_fullsized":"2024-05-11T01:01:02.913695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:04:19.015364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676868+0000","last_change":"2024-05-11T01:01:01.690633+0000","last_active":"2024-05-11T01:01:02.676868+0000","last_peered":"2024-05-11T01:01:02.676868+0000","last_clean":"2024-05-11T01:01:02.676868+0000","last_became_active":"2024-05-11T01:01:01.690227+0000","last_became_peered":"2024-05-11T01:01:01.690227+0000","last_unstale":"2024-05-11T01:01:02.676868+0000","last_undegraded":"2024-05-11T01:01:02.676868+0000","last_fullsized":"2024-05-11T01:01:02.676868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:24.703345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678131+0000","last_change":"2024-05-11T01:01:01.688827+0000","last_active":"2024-05-11T01:01:02.678131+0000","last_peered":"2024-05-11T01:01:02.678131+0000","last_clean":"2024-05-11T01:01:02.678131+0000","last_became_active":"2024-05-11T01:01:01.688376+0000","last_became_peered":"2024-05-11T01:01:01.688376+0000","last_unstale":"2024-05-11T01:01:02.678131+0000","last_undegraded":"2024-05-11T01:01:02.678131+0000","last_fullsized":"2024-05-11T01:01:02.678131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:24.858093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912912+0000","last_change":"2024-05-11T01:01:01.692633+0000","last_active":"2024-05-11T01:01:02.912912+0000","last_peered":"2024-05-11T01:01:02.912912+0000","last_clean":"2024-05-11T01:01:02.912912+0000","last_became_active":"2024-05-11T01:01:01.692305+0000","last_became_peered":"2024-05-11T01:01:01.692305+0000","last_unstale":"2024-05-11T01:01:02.912912+0000","last_undegraded":"2024-05-11T01:01:02.912912+0000","last_fullsized":"2024-05-11T01:01:02.912912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:10:34.198396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705339+0000","last_change":"2024-05-11T01:01:01.692423+0000","last_active":"2024-05-11T01:01:02.705339+0000","last_peered":"2024-05-11T01:01:02.705339+0000","last_clean":"2024-05-11T01:01:02.705339+0000","last_became_active":"2024-05-11T01:01:01.691669+0000","last_became_peered":"2024-05-11T01:01:01.691669+0000","last_unstale":"2024-05-11T01:01:02.705339+0000","last_undegraded":"2024-05-11T01:01:02.705339+0000","last_fullsized":"2024-05-11T01:01:02.705339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:10:39.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677711+0000","last_change":"2024-05-11T01:01:01.691243+0000","last_active":"2024-05-11T01:01:02.677711+0000","last_peered":"2024-05-11T01:01:02.677711+0000","last_clean":"2024-05-11T01:01:02.677711+0000","last_became_active":"2024-05-11T01:01:01.690974+0000","last_became_peered":"2024-05-11T01:01:01.690974+0000","last_unstale":"2024-05-11T01:01:02.677711+0000","last_undegraded":"2024-05-11T01:01:02.677711+0000","last_fullsized":"2024-05-11T01:01:02.677711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:25.558002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913441+0000","last_change":"2024-05-11T01:01:00.681960+0000","last_active":"2024-05-11T01:01:02.913441+0000","last_peered":"2024-05-11T01:01:02.913441+0000","last_clean":"2024-05-11T01:01:02.913441+0000","last_became_active":"2024-05-11T01:01:00.681658+0000","last_became_peered":"2024-05-11T01:01:00.681658+0000","last_unstale":"2024-05-11T01:01:02.913441+0000","last_undegraded":"2024-05-11T01:01:02.913441+0000","last_fullsized":"2024-05-11T01:01:02.913441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676953+0000","last_change":"2024-05-11T01:01:01.697201+0000","last_active":"2024-05-11T01:01:02.676953+0000","last_peered":"2024-05-11T01:01:02.676953+0000","last_clean":"2024-05-11T01:01:02.676953+0000","last_became_active":"2024-05-11T01:01:01.696525+0000","last_became_peered":"2024-05-11T01:01:01.696525+0000","last_unstale":"2024-05-11T01:01:02.676953+0000","last_undegraded":"2024-05-11T01:01:02.676953+0000","last_fullsized":"2024-05-11T01:01:02.676953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:27:35.463648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705370+0000","last_change":"2024-05-11T01:01:00.684578+0000","last_active":"2024-05-11T01:01:02.705370+0000","last_peered":"2024-05-11T01:01:02.705370+0000","last_clean":"2024-05-11T01:01:02.705370+0000","last_became_active":"2024-05-11T01:01:00.684303+0000","last_became_peered":"2024-05-11T01:01:00.684303+0000","last_unstale":"2024-05-11T01:01:02.705370+0000","last_undegraded":"2024-05-11T01:01:02.705370+0000","last_fullsized":"2024-05-11T01:01:02.705370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913278+0000","last_change":"2024-05-11T01:01:00.678144+0000","last_active":"2024-05-11T01:01:02.913278+0000","last_peered":"2024-05-11T01:01:02.913278+0000","last_clean":"2024-05-11T01:01:02.913278+0000","last_became_active":"2024-05-11T01:01:00.677790+0000","last_became_peered":"2024-05-11T01:01:00.677790+0000","last_unstale":"2024-05-11T01:01:02.913278+0000","last_undegraded":"2024-05-11T01:01:02.913278+0000","last_fullsized":"2024-05-11T01:01:02.913278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:38:37.156429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913068+0000","last_change":"2024-05-11T01:01:01.689344+0000","last_active":"2024-05-11T01:01:02.913068+0000","last_peered":"2024-05-11T01:01:02.913068+0000","last_clean":"2024-05-11T01:01:02.913068+0000","last_became_active":"2024-05-11T01:01:01.687984+0000","last_became_peered":"2024-05-11T01:01:01.687984+0000","last_unstale":"2024-05-11T01:01:02.913068+0000","last_undegraded":"2024-05-11T01:01:02.913068+0000","last_fullsized":"2024-05-11T01:01:02.913068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:27:35.463648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678187+0000","last_change":"2024-05-11T01:01:00.678981+0000","last_active":"2024-05-11T01:01:02.678187+0000","last_peered":"2024-05-11T01:01:02.678187+0000","last_clean":"2024-05-11T01:01:02.678187+0000","last_became_active":"2024-05-11T01:01:00.678672+0000","last_became_peered":"2024-05-11T01:01:00.678672+0000","last_unstale":"2024-05-11T01:01:02.678187+0000","last_undegraded":"2024-05-11T01:01:02.678187+0000","last_fullsized":"2024-05-11T01:01:02.678187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.702964+0000","last_change":"2024-05-11T01:01:01.694506+0000","last_active":"2024-05-11T01:01:02.702964+0000","last_peered":"2024-05-11T01:01:02.702964+0000","last_clean":"2024-05-11T01:01:02.702964+0000","last_became_active":"2024-05-11T01:01:01.694226+0000","last_became_peered":"2024-05-11T01:01:01.694226+0000","last_unstale":"2024-05-11T01:01:02.702964+0000","last_undegraded":"2024-05-11T01:01:02.702964+0000","last_fullsized":"2024-05-11T01:01:02.702964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705643+0000","last_change":"2024-05-11T01:01:00.679681+0000","last_active":"2024-05-11T01:01:02.705643+0000","last_peered":"2024-05-11T01:01:02.705643+0000","last_clean":"2024-05-11T01:01:02.705643+0000","last_became_active":"2024-05-11T01:01:00.679125+0000","last_became_peered":"2024-05-11T01:01:00.679125+0000","last_unstale":"2024-05-11T01:01:02.705643+0000","last_undegraded":"2024-05-11T01:01:02.705643+0000","last_fullsized":"2024-05-11T01:01:02.705643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678282+0000","last_change":"2024-05-11T01:01:01.690424+0000","last_active":"2024-05-11T01:01:02.678282+0000","last_peered":"2024-05-11T01:01:02.678282+0000","last_clean":"2024-05-11T01:01:02.678282+0000","last_became_active":"2024-05-11T01:01:01.690131+0000","last_became_peered":"2024-05-11T01:01:01.690131+0000","last_unstale":"2024-05-11T01:01:02.678282+0000","last_undegraded":"2024-05-11T01:01:02.678282+0000","last_fullsized":"2024-05-11T01:01:02.678282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:35:54.225824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678389+0000","last_change":"2024-05-11T01:01:00.676347+0000","last_active":"2024-05-11T01:01:02.678389+0000","last_peered":"2024-05-11T01:01:02.678389+0000","last_clean":"2024-05-11T01:01:02.678389+0000","last_became_active":"2024-05-11T01:01:00.676058+0000","last_became_peered":"2024-05-11T01:01:00.676058+0000","last_unstale":"2024-05-11T01:01:02.678389+0000","last_undegraded":"2024-05-11T01:01:02.678389+0000","last_fullsized":"2024-05-11T01:01:02.678389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677252+0000","last_change":"2024-05-11T01:01:01.685176+0000","last_active":"2024-05-11T01:01:02.677252+0000","last_peered":"2024-05-11T01:01:02.677252+0000","last_clean":"2024-05-11T01:01:02.677252+0000","last_became_active":"2024-05-11T01:01:01.684804+0000","last_became_peered":"2024-05-11T01:01:01.684804+0000","last_unstale":"2024-05-11T01:01:02.677252+0000","last_undegraded":"2024-05-11T01:01:02.677252+0000","last_fullsized":"2024-05-11T01:01:02.677252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703190+0000","last_change":"2024-05-11T01:01:00.673845+0000","last_active":"2024-05-11T01:01:02.703190+0000","last_peered":"2024-05-11T01:01:02.703190+0000","last_clean":"2024-05-11T01:01:02.703190+0000","last_became_active":"2024-05-11T01:01:00.673430+0000","last_became_peered":"2024-05-11T01:01:00.673430+0000","last_unstale":"2024-05-11T01:01:02.703190+0000","last_undegraded":"2024-05-11T01:01:02.703190+0000","last_fullsized":"2024-05-11T01:01:02.703190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678463+0000","last_change":"2024-05-11T01:01:01.695552+0000","last_active":"2024-05-11T01:01:02.678463+0000","last_peered":"2024-05-11T01:01:02.678463+0000","last_clean":"2024-05-11T01:01:02.678463+0000","last_became_active":"2024-05-11T01:01:01.693674+0000","last_became_peered":"2024-05-11T01:01:01.693674+0000","last_unstale":"2024-05-11T01:01:02.678463+0000","last_undegraded":"2024-05-11T01:01:02.678463+0000","last_fullsized":"2024-05-11T01:01:02.678463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:17.700359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705505+0000","last_change":"2024-05-11T01:01:00.683575+0000","last_active":"2024-05-11T01:01:02.705505+0000","last_peered":"2024-05-11T01:01:02.705505+0000","last_clean":"2024-05-11T01:01:02.705505+0000","last_became_active":"2024-05-11T01:01:00.683259+0000","last_became_peered":"2024-05-11T01:01:00.683259+0000","last_unstale":"2024-05-11T01:01:02.705505+0000","last_undegraded":"2024-05-11T01:01:02.705505+0000","last_fullsized":"2024-05-11T01:01:02.705505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793910+0000","last_change":"2024-05-11T01:01:01.699906+0000","last_active":"2024-05-11T01:01:02.793910+0000","last_peered":"2024-05-11T01:01:02.793910+0000","last_clean":"2024-05-11T01:01:02.793910+0000","last_became_active":"2024-05-11T01:01:01.699637+0000","last_became_peered":"2024-05-11T01:01:01.699637+0000","last_unstale":"2024-05-11T01:01:02.793910+0000","last_undegraded":"2024-05-11T01:01:02.793910+0000","last_fullsized":"2024-05-11T01:01:02.793910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:13:49.063574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677033+0000","last_change":"2024-05-11T01:01:00.684668+0000","last_active":"2024-05-11T01:01:02.677033+0000","last_peered":"2024-05-11T01:01:02.677033+0000","last_clean":"2024-05-11T01:01:02.677033+0000","last_became_active":"2024-05-11T01:01:00.684364+0000","last_became_peered":"2024-05-11T01:01:00.684364+0000","last_unstale":"2024-05-11T01:01:02.677033+0000","last_undegraded":"2024-05-11T01:01:02.677033+0000","last_fullsized":"2024-05-11T01:01:02.677033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:54.494836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676757+0000","last_change":"2024-05-11T01:01:01.686608+0000","last_active":"2024-05-11T01:01:02.676757+0000","last_peered":"2024-05-11T01:01:02.676757+0000","last_clean":"2024-05-11T01:01:02.676757+0000","last_became_active":"2024-05-11T01:01:01.686333+0000","last_became_peered":"2024-05-11T01:01:01.686333+0000","last_unstale":"2024-05-11T01:01:02.676757+0000","last_undegraded":"2024-05-11T01:01:02.676757+0000","last_fullsized":"2024-05-11T01:01:02.676757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:51.874581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678282+0000","last_change":"2024-05-11T01:01:00.684592+0000","last_active":"2024-05-11T01:01:02.678282+0000","last_peered":"2024-05-11T01:01:02.678282+0000","last_clean":"2024-05-11T01:01:02.678282+0000","last_became_active":"2024-05-11T01:01:00.683709+0000","last_became_peered":"2024-05-11T01:01:00.683709+0000","last_unstale":"2024-05-11T01:01:02.678282+0000","last_undegraded":"2024-05-11T01:01:02.678282+0000","last_fullsized":"2024-05-11T01:01:02.678282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793773+0000","last_change":"2024-05-11T01:01:01.699198+0000","last_active":"2024-05-11T01:01:02.793773+0000","last_peered":"2024-05-11T01:01:02.793773+0000","last_clean":"2024-05-11T01:01:02.793773+0000","last_became_active":"2024-05-11T01:01:01.698959+0000","last_became_peered":"2024-05-11T01:01:01.698959+0000","last_unstale":"2024-05-11T01:01:02.793773+0000","last_undegraded":"2024-05-11T01:01:02.793773+0000","last_fullsized":"2024-05-11T01:01:02.793773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:32.857796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705344+0000","last_change":"2024-05-11T01:01:00.684714+0000","last_active":"2024-05-11T01:01:02.705344+0000","last_peered":"2024-05-11T01:01:02.705344+0000","last_clean":"2024-05-11T01:01:02.705344+0000","last_became_active":"2024-05-11T01:01:00.684454+0000","last_became_peered":"2024-05-11T01:01:00.684454+0000","last_unstale":"2024-05-11T01:01:02.705344+0000","last_undegraded":"2024-05-11T01:01:02.705344+0000","last_fullsized":"2024-05-11T01:01:02.705344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705449+0000","last_change":"2024-05-11T01:01:01.691666+0000","last_active":"2024-05-11T01:01:02.705449+0000","last_peered":"2024-05-11T01:01:02.705449+0000","last_clean":"2024-05-11T01:01:02.705449+0000","last_became_active":"2024-05-11T01:01:01.691368+0000","last_became_peered":"2024-05-11T01:01:01.691368+0000","last_unstale":"2024-05-11T01:01:02.705449+0000","last_undegraded":"2024-05-11T01:01:02.705449+0000","last_fullsized":"2024-05-11T01:01:02.705449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912707+0000","last_change":"2024-05-11T01:01:00.680506+0000","last_active":"2024-05-11T01:01:02.912707+0000","last_peered":"2024-05-11T01:01:02.912707+0000","last_clean":"2024-05-11T01:01:02.912707+0000","last_became_active":"2024-05-11T01:01:00.680198+0000","last_became_peered":"2024-05-11T01:01:00.680198+0000","last_unstale":"2024-05-11T01:01:02.912707+0000","last_undegraded":"2024-05-11T01:01:02.912707+0000","last_fullsized":"2024-05-11T01:01:02.912707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793544+0000","last_change":"2024-05-11T01:01:01.698747+0000","last_active":"2024-05-11T01:01:02.793544+0000","last_peered":"2024-05-11T01:01:02.793544+0000","last_clean":"2024-05-11T01:01:02.793544+0000","last_became_active":"2024-05-11T01:01:01.698482+0000","last_became_peered":"2024-05-11T01:01:01.698482+0000","last_unstale":"2024-05-11T01:01:02.793544+0000","last_undegraded":"2024-05-11T01:01:02.793544+0000","last_fullsized":"2024-05-11T01:01:02.793544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:35:54.225824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678878+0000","last_change":"2024-05-11T01:01:00.686573+0000","last_active":"2024-05-11T01:01:02.678878+0000","last_peered":"2024-05-11T01:01:02.678878+0000","last_clean":"2024-05-11T01:01:02.678878+0000","last_became_active":"2024-05-11T01:01:00.686222+0000","last_became_peered":"2024-05-11T01:01:00.686222+0000","last_unstale":"2024-05-11T01:01:02.678878+0000","last_undegraded":"2024-05-11T01:01:02.678878+0000","last_fullsized":"2024-05-11T01:01:02.678878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:20:07.656810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794105+0000","last_change":"2024-05-11T01:01:01.697983+0000","last_active":"2024-05-11T01:01:02.794105+0000","last_peered":"2024-05-11T01:01:02.794105+0000","last_clean":"2024-05-11T01:01:02.794105+0000","last_became_active":"2024-05-11T01:01:01.697645+0000","last_became_peered":"2024-05-11T01:01:01.697645+0000","last_unstale":"2024-05-11T01:01:02.794105+0000","last_undegraded":"2024-05-11T01:01:02.794105+0000","last_fullsized":"2024-05-11T01:01:02.794105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:12.996809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913020+0000","last_change":"2024-05-11T01:01:00.684168+0000","last_active":"2024-05-11T01:01:02.913020+0000","last_peered":"2024-05-11T01:01:02.913020+0000","last_clean":"2024-05-11T01:01:02.913020+0000","last_became_active":"2024-05-11T01:01:00.683839+0000","last_became_peered":"2024-05-11T01:01:00.683839+0000","last_unstale":"2024-05-11T01:01:02.913020+0000","last_undegraded":"2024-05-11T01:01:02.913020+0000","last_fullsized":"2024-05-11T01:01:02.913020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794232+0000","last_change":"2024-05-11T01:01:01.698302+0000","last_active":"2024-05-11T01:01:02.794232+0000","last_peered":"2024-05-11T01:01:02.794232+0000","last_clean":"2024-05-11T01:01:02.794232+0000","last_became_active":"2024-05-11T01:01:01.697989+0000","last_became_peered":"2024-05-11T01:01:01.697989+0000","last_unstale":"2024-05-11T01:01:02.794232+0000","last_undegraded":"2024-05-11T01:01:02.794232+0000","last_fullsized":"2024-05-11T01:01:02.794232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:20:37.893643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703231+0000","last_change":"2024-05-11T01:01:00.681331+0000","last_active":"2024-05-11T01:01:02.703231+0000","last_peered":"2024-05-11T01:01:02.703231+0000","last_clean":"2024-05-11T01:01:02.703231+0000","last_became_active":"2024-05-11T01:01:00.680679+0000","last_became_peered":"2024-05-11T01:01:00.680679+0000","last_unstale":"2024-05-11T01:01:02.703231+0000","last_undegraded":"2024-05-11T01:01:02.703231+0000","last_fullsized":"2024-05-11T01:01:02.703231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678940+0000","last_change":"2024-05-11T01:01:01.687504+0000","last_active":"2024-05-11T01:01:02.678940+0000","last_peered":"2024-05-11T01:01:02.678940+0000","last_clean":"2024-05-11T01:01:02.678940+0000","last_became_active":"2024-05-11T01:01:01.686851+0000","last_became_peered":"2024-05-11T01:01:01.686851+0000","last_unstale":"2024-05-11T01:01:02.678940+0000","last_undegraded":"2024-05-11T01:01:02.678940+0000","last_fullsized":"2024-05-11T01:01:02.678940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:00:03.997273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913094+0000","last_change":"2024-05-11T01:01:00.682797+0000","last_active":"2024-05-11T01:01:02.913094+0000","last_peered":"2024-05-11T01:01:02.913094+0000","last_clean":"2024-05-11T01:01:02.913094+0000","last_became_active":"2024-05-11T01:01:00.682282+0000","last_became_peered":"2024-05-11T01:01:00.682282+0000","last_unstale":"2024-05-11T01:01:02.913094+0000","last_undegraded":"2024-05-11T01:01:02.913094+0000","last_fullsized":"2024-05-11T01:01:02.913094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:00:17.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793948+0000","last_change":"2024-05-11T01:01:01.705606+0000","last_active":"2024-05-11T01:01:02.793948+0000","last_peered":"2024-05-11T01:01:02.793948+0000","last_clean":"2024-05-11T01:01:02.793948+0000","last_became_active":"2024-05-11T01:01:01.705324+0000","last_became_peered":"2024-05-11T01:01:01.705324+0000","last_unstale":"2024-05-11T01:01:02.793948+0000","last_undegraded":"2024-05-11T01:01:02.793948+0000","last_fullsized":"2024-05-11T01:01:02.793948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:31.889721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793424+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:01:02.793424+0000","last_peered":"2024-05-11T01:01:02.793424+0000","last_clean":"2024-05-11T01:01:02.793424+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:01:02.793424+0000","last_undegraded":"2024-05-11T01:01:02.793424+0000","last_fullsized":"2024-05-11T01:01:02.793424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677503+0000","last_change":"2024-05-11T01:01:00.679561+0000","last_active":"2024-05-11T01:01:02.677503+0000","last_peered":"2024-05-11T01:01:02.677503+0000","last_clean":"2024-05-11T01:01:02.677503+0000","last_became_active":"2024-05-11T01:01:00.679241+0000","last_became_peered":"2024-05-11T01:01:00.679241+0000","last_unstale":"2024-05-11T01:01:02.677503+0000","last_undegraded":"2024-05-11T01:01:02.677503+0000","last_fullsized":"2024-05-11T01:01:02.677503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913407+0000","last_change":"2024-05-11T01:01:01.693541+0000","last_active":"2024-05-11T01:01:02.913407+0000","last_peered":"2024-05-11T01:01:02.913407+0000","last_clean":"2024-05-11T01:01:02.913407+0000","last_became_active":"2024-05-11T01:01:01.693033+0000","last_became_peered":"2024-05-11T01:01:01.693033+0000","last_unstale":"2024-05-11T01:01:02.913407+0000","last_undegraded":"2024-05-11T01:01:02.913407+0000","last_fullsized":"2024-05-11T01:01:02.913407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793351+0000","last_change":"2024-05-11T01:01:00.689701+0000","last_active":"2024-05-11T01:01:02.793351+0000","last_peered":"2024-05-11T01:01:02.793351+0000","last_clean":"2024-05-11T01:01:02.793351+0000","last_became_active":"2024-05-11T01:01:00.689439+0000","last_became_peered":"2024-05-11T01:01:00.689439+0000","last_unstale":"2024-05-11T01:01:02.793351+0000","last_undegraded":"2024-05-11T01:01:02.793351+0000","last_fullsized":"2024-05-11T01:01:02.793351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:10:33.194315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676839+0000","last_change":"2024-05-11T01:01:01.696524+0000","last_active":"2024-05-11T01:01:02.676839+0000","last_peered":"2024-05-11T01:01:02.676839+0000","last_clean":"2024-05-11T01:01:02.676839+0000","last_became_active":"2024-05-11T01:01:01.696245+0000","last_became_peered":"2024-05-11T01:01:01.696245+0000","last_unstale":"2024-05-11T01:01:02.676839+0000","last_undegraded":"2024-05-11T01:01:02.676839+0000","last_fullsized":"2024-05-11T01:01:02.676839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:33.546403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912909+0000","last_change":"2024-05-11T01:01:00.683932+0000","last_active":"2024-05-11T01:01:02.912909+0000","last_peered":"2024-05-11T01:01:02.912909+0000","last_clean":"2024-05-11T01:01:02.912909+0000","last_became_active":"2024-05-11T01:01:00.683601+0000","last_became_peered":"2024-05-11T01:01:00.683601+0000","last_unstale":"2024-05-11T01:01:02.912909+0000","last_undegraded":"2024-05-11T01:01:02.912909+0000","last_fullsized":"2024-05-11T01:01:02.912909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.702979+0000","last_change":"2024-05-11T01:01:01.691231+0000","last_active":"2024-05-11T01:01:02.702979+0000","last_peered":"2024-05-11T01:01:02.702979+0000","last_clean":"2024-05-11T01:01:02.702979+0000","last_became_active":"2024-05-11T01:01:01.690969+0000","last_became_peered":"2024-05-11T01:01:01.690969+0000","last_unstale":"2024-05-11T01:01:02.702979+0000","last_undegraded":"2024-05-11T01:01:02.702979+0000","last_fullsized":"2024-05-11T01:01:02.702979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678402+0000","last_change":"2024-05-11T01:01:00.682079+0000","last_active":"2024-05-11T01:01:02.678402+0000","last_peered":"2024-05-11T01:01:02.678402+0000","last_clean":"2024-05-11T01:01:02.678402+0000","last_became_active":"2024-05-11T01:01:00.681773+0000","last_became_peered":"2024-05-11T01:01:00.681773+0000","last_unstale":"2024-05-11T01:01:02.678402+0000","last_undegraded":"2024-05-11T01:01:02.678402+0000","last_fullsized":"2024-05-11T01:01:02.678402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.679107+0000","last_change":"2024-05-11T01:01:01.695539+0000","last_active":"2024-05-11T01:01:02.679107+0000","last_peered":"2024-05-11T01:01:02.679107+0000","last_clean":"2024-05-11T01:01:02.679107+0000","last_became_active":"2024-05-11T01:01:01.695235+0000","last_became_peered":"2024-05-11T01:01:01.695235+0000","last_unstale":"2024-05-11T01:01:02.679107+0000","last_undegraded":"2024-05-11T01:01:02.679107+0000","last_fullsized":"2024-05-11T01:01:02.679107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:44.383036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703144+0000","last_change":"2024-05-11T01:01:00.680453+0000","last_active":"2024-05-11T01:01:02.703144+0000","last_peered":"2024-05-11T01:01:02.703144+0000","last_clean":"2024-05-11T01:01:02.703144+0000","last_became_active":"2024-05-11T01:01:00.679559+0000","last_became_peered":"2024-05-11T01:01:00.679559+0000","last_unstale":"2024-05-11T01:01:02.703144+0000","last_undegraded":"2024-05-11T01:01:02.703144+0000","last_fullsized":"2024-05-11T01:01:02.703144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:10:33.194315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678735+0000","last_change":"2024-05-11T01:01:01.687198+0000","last_active":"2024-05-11T01:01:02.678735+0000","last_peered":"2024-05-11T01:01:02.678735+0000","last_clean":"2024-05-11T01:01:02.678735+0000","last_became_active":"2024-05-11T01:01:01.686860+0000","last_became_peered":"2024-05-11T01:01:01.686860+0000","last_unstale":"2024-05-11T01:01:02.678735+0000","last_undegraded":"2024-05-11T01:01:02.678735+0000","last_fullsized":"2024-05-11T01:01:02.678735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:31.889721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794088+0000","last_change":"2024-05-11T01:01:00.688488+0000","last_active":"2024-05-11T01:01:02.794088+0000","last_peered":"2024-05-11T01:01:02.794088+0000","last_clean":"2024-05-11T01:01:02.794088+0000","last_became_active":"2024-05-11T01:01:00.688109+0000","last_became_peered":"2024-05-11T01:01:00.688109+0000","last_unstale":"2024-05-11T01:01:02.794088+0000","last_undegraded":"2024-05-11T01:01:02.794088+0000","last_fullsized":"2024-05-11T01:01:02.794088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:32.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703144+0000","last_change":"2024-05-11T01:01:01.689340+0000","last_active":"2024-05-11T01:01:02.703144+0000","last_peered":"2024-05-11T01:01:02.703144+0000","last_clean":"2024-05-11T01:01:02.703144+0000","last_became_active":"2024-05-11T01:01:01.689052+0000","last_became_peered":"2024-05-11T01:01:01.689052+0000","last_unstale":"2024-05-11T01:01:02.703144+0000","last_undegraded":"2024-05-11T01:01:02.703144+0000","last_fullsized":"2024-05-11T01:01:02.703144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703207+0000","last_change":"2024-05-11T01:01:00.678958+0000","last_active":"2024-05-11T01:01:02.703207+0000","last_peered":"2024-05-11T01:01:02.703207+0000","last_clean":"2024-05-11T01:01:02.703207+0000","last_became_active":"2024-05-11T01:01:00.678608+0000","last_became_peered":"2024-05-11T01:01:00.678608+0000","last_unstale":"2024-05-11T01:01:02.703207+0000","last_undegraded":"2024-05-11T01:01:02.703207+0000","last_fullsized":"2024-05-11T01:01:02.703207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:58.764462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913742+0000","last_change":"2024-05-11T01:01:01.686046+0000","last_active":"2024-05-11T01:01:02.913742+0000","last_peered":"2024-05-11T01:01:02.913742+0000","last_clean":"2024-05-11T01:01:02.913742+0000","last_became_active":"2024-05-11T01:01:01.685771+0000","last_became_peered":"2024-05-11T01:01:01.685771+0000","last_unstale":"2024-05-11T01:01:02.913742+0000","last_undegraded":"2024-05-11T01:01:02.913742+0000","last_fullsized":"2024-05-11T01:01:02.913742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:36:33.062657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705667+0000","last_change":"2024-05-11T01:01:00.686048+0000","last_active":"2024-05-11T01:01:02.705667+0000","last_peered":"2024-05-11T01:01:02.705667+0000","last_clean":"2024-05-11T01:01:02.705667+0000","last_became_active":"2024-05-11T01:01:00.685215+0000","last_became_peered":"2024-05-11T01:01:00.685215+0000","last_unstale":"2024-05-11T01:01:02.705667+0000","last_undegraded":"2024-05-11T01:01:02.705667+0000","last_fullsized":"2024-05-11T01:01:02.705667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677114+0000","last_change":"2024-05-11T01:01:01.695783+0000","last_active":"2024-05-11T01:01:02.677114+0000","last_peered":"2024-05-11T01:01:02.677114+0000","last_clean":"2024-05-11T01:01:02.677114+0000","last_became_active":"2024-05-11T01:01:01.695264+0000","last_became_peered":"2024-05-11T01:01:01.695264+0000","last_unstale":"2024-05-11T01:01:02.677114+0000","last_undegraded":"2024-05-11T01:01:02.677114+0000","last_fullsized":"2024-05-11T01:01:02.677114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793321+0000","last_change":"2024-05-11T01:01:00.698030+0000","last_active":"2024-05-11T01:01:02.793321+0000","last_peered":"2024-05-11T01:01:02.793321+0000","last_clean":"2024-05-11T01:01:02.793321+0000","last_became_active":"2024-05-11T01:01:00.697776+0000","last_became_peered":"2024-05-11T01:01:00.697776+0000","last_unstale":"2024-05-11T01:01:02.793321+0000","last_undegraded":"2024-05-11T01:01:02.793321+0000","last_fullsized":"2024-05-11T01:01:02.793321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677953+0000","last_change":"2024-05-11T01:01:01.696098+0000","last_active":"2024-05-11T01:01:02.677953+0000","last_peered":"2024-05-11T01:01:02.677953+0000","last_clean":"2024-05-11T01:01:02.677953+0000","last_became_active":"2024-05-11T01:01:01.695670+0000","last_became_peered":"2024-05-11T01:01:01.695670+0000","last_unstale":"2024-05-11T01:01:02.677953+0000","last_undegraded":"2024-05-11T01:01:02.677953+0000","last_fullsized":"2024-05-11T01:01:02.677953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:49.477561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703023+0000","last_change":"2024-05-11T01:01:00.680814+0000","last_active":"2024-05-11T01:01:02.703023+0000","last_peered":"2024-05-11T01:01:02.703023+0000","last_clean":"2024-05-11T01:01:02.703023+0000","last_became_active":"2024-05-11T01:01:00.680415+0000","last_became_peered":"2024-05-11T01:01:00.680415+0000","last_unstale":"2024-05-11T01:01:02.703023+0000","last_undegraded":"2024-05-11T01:01:02.703023+0000","last_fullsized":"2024-05-11T01:01:02.703023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913100+0000","last_change":"2024-05-11T01:01:01.690240+0000","last_active":"2024-05-11T01:01:02.913100+0000","last_peered":"2024-05-11T01:01:02.913100+0000","last_clean":"2024-05-11T01:01:02.913100+0000","last_became_active":"2024-05-11T01:01:01.689938+0000","last_became_peered":"2024-05-11T01:01:01.689938+0000","last_unstale":"2024-05-11T01:01:02.913100+0000","last_undegraded":"2024-05-11T01:01:02.913100+0000","last_fullsized":"2024-05-11T01:01:02.913100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:10:39.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703036+0000","last_change":"2024-05-11T01:01:00.682185+0000","last_active":"2024-05-11T01:01:02.703036+0000","last_peered":"2024-05-11T01:01:02.703036+0000","last_clean":"2024-05-11T01:01:02.703036+0000","last_became_active":"2024-05-11T01:01:00.681773+0000","last_became_peered":"2024-05-11T01:01:00.681773+0000","last_unstale":"2024-05-11T01:01:02.703036+0000","last_undegraded":"2024-05-11T01:01:02.703036+0000","last_fullsized":"2024-05-11T01:01:02.703036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913467+0000","last_change":"2024-05-11T01:01:01.688107+0000","last_active":"2024-05-11T01:01:02.913467+0000","last_peered":"2024-05-11T01:01:02.913467+0000","last_clean":"2024-05-11T01:01:02.913467+0000","last_became_active":"2024-05-11T01:01:01.686201+0000","last_became_peered":"2024-05-11T01:01:01.686201+0000","last_unstale":"2024-05-11T01:01:02.913467+0000","last_undegraded":"2024-05-11T01:01:02.913467+0000","last_fullsized":"2024-05-11T01:01:02.913467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793707+0000","last_change":"2024-05-11T01:01:00.683637+0000","last_active":"2024-05-11T01:01:02.793707+0000","last_peered":"2024-05-11T01:01:02.793707+0000","last_clean":"2024-05-11T01:01:02.793707+0000","last_became_active":"2024-05-11T01:01:00.683373+0000","last_became_peered":"2024-05-11T01:01:00.683373+0000","last_unstale":"2024-05-11T01:01:02.793707+0000","last_undegraded":"2024-05-11T01:01:02.793707+0000","last_fullsized":"2024-05-11T01:01:02.793707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:58.764462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705465+0000","last_change":"2024-05-11T01:01:01.695407+0000","last_active":"2024-05-11T01:01:02.705465+0000","last_peered":"2024-05-11T01:01:02.705465+0000","last_clean":"2024-05-11T01:01:02.705465+0000","last_became_active":"2024-05-11T01:01:01.695012+0000","last_became_peered":"2024-05-11T01:01:01.695012+0000","last_unstale":"2024-05-11T01:01:02.705465+0000","last_undegraded":"2024-05-11T01:01:02.705465+0000","last_fullsized":"2024-05-11T01:01:02.705465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678475+0000","last_change":"2024-05-11T01:01:01.698832+0000","last_active":"2024-05-11T01:01:02.678475+0000","last_peered":"2024-05-11T01:01:02.678475+0000","last_clean":"2024-05-11T01:01:02.678475+0000","last_became_active":"2024-05-11T01:01:01.698487+0000","last_became_peered":"2024-05-11T01:01:01.698487+0000","last_unstale":"2024-05-11T01:01:02.678475+0000","last_undegraded":"2024-05-11T01:01:02.678475+0000","last_fullsized":"2024-05-11T01:01:02.678475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793796+0000","last_change":"2024-05-11T01:01:00.690097+0000","last_active":"2024-05-11T01:01:02.793796+0000","last_peered":"2024-05-11T01:01:02.793796+0000","last_clean":"2024-05-11T01:01:02.793796+0000","last_became_active":"2024-05-11T01:01:00.689772+0000","last_became_peered":"2024-05-11T01:01:00.689772+0000","last_unstale":"2024-05-11T01:01:02.793796+0000","last_undegraded":"2024-05-11T01:01:02.793796+0000","last_fullsized":"2024-05-11T01:01:02.793796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678776+0000","last_change":"2024-05-11T01:01:01.699646+0000","last_active":"2024-05-11T01:01:02.678776+0000","last_peered":"2024-05-11T01:01:02.678776+0000","last_clean":"2024-05-11T01:01:02.678776+0000","last_became_active":"2024-05-11T01:01:01.699388+0000","last_became_peered":"2024-05-11T01:01:01.699388+0000","last_unstale":"2024-05-11T01:01:02.678776+0000","last_undegraded":"2024-05-11T01:01:02.678776+0000","last_fullsized":"2024-05-11T01:01:02.678776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:12.996809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793982+0000","last_change":"2024-05-11T01:01:00.688793+0000","last_active":"2024-05-11T01:01:02.793982+0000","last_peered":"2024-05-11T01:01:02.793982+0000","last_clean":"2024-05-11T01:01:02.793982+0000","last_became_active":"2024-05-11T01:01:00.688279+0000","last_became_peered":"2024-05-11T01:01:00.688279+0000","last_unstale":"2024-05-11T01:01:02.793982+0000","last_undegraded":"2024-05-11T01:01:02.793982+0000","last_fullsized":"2024-05-11T01:01:02.793982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:50.870500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677179+0000","last_change":"2024-05-11T01:01:01.682113+0000","last_active":"2024-05-11T01:01:02.677179+0000","last_peered":"2024-05-11T01:01:02.677179+0000","last_clean":"2024-05-11T01:01:02.677179+0000","last_became_active":"2024-05-11T01:01:01.681706+0000","last_became_peered":"2024-05-11T01:01:01.681706+0000","last_unstale":"2024-05-11T01:01:02.677179+0000","last_undegraded":"2024-05-11T01:01:02.677179+0000","last_fullsized":"2024-05-11T01:01:02.677179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913646+0000","last_change":"2024-05-11T01:01:00.680118+0000","last_active":"2024-05-11T01:01:02.913646+0000","last_peered":"2024-05-11T01:01:02.913646+0000","last_clean":"2024-05-11T01:01:02.913646+0000","last_became_active":"2024-05-11T01:01:00.679809+0000","last_became_peered":"2024-05-11T01:01:00.679809+0000","last_unstale":"2024-05-11T01:01:02.913646+0000","last_undegraded":"2024-05-11T01:01:02.913646+0000","last_fullsized":"2024-05-11T01:01:02.913646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:12:31.764619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913693+0000","last_change":"2024-05-11T01:01:01.692697+0000","last_active":"2024-05-11T01:01:02.913693+0000","last_peered":"2024-05-11T01:01:02.913693+0000","last_clean":"2024-05-11T01:01:02.913693+0000","last_became_active":"2024-05-11T01:01:01.691026+0000","last_became_peered":"2024-05-11T01:01:01.691026+0000","last_unstale":"2024-05-11T01:01:02.913693+0000","last_undegraded":"2024-05-11T01:01:02.913693+0000","last_fullsized":"2024-05-11T01:01:02.913693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677208+0000","last_change":"2024-05-11T01:01:00.690191+0000","last_active":"2024-05-11T01:01:02.677208+0000","last_peered":"2024-05-11T01:01:02.677208+0000","last_clean":"2024-05-11T01:01:02.677208+0000","last_became_active":"2024-05-11T01:01:00.689804+0000","last_became_peered":"2024-05-11T01:01:00.689804+0000","last_unstale":"2024-05-11T01:01:02.677208+0000","last_undegraded":"2024-05-11T01:01:02.677208+0000","last_fullsized":"2024-05-11T01:01:02.677208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:13.659371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:01:14.799 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.939+0000 7f8fee504640 1 Processor -- start 2024-05-11T01:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.939+0000 7f8fee504640 1 -- start start 2024-05-11T01:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.940+0000 7f8fee504640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80b5dd0 0x7f8fe80d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.940+0000 7f8fee504640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe8163250 con 0x7f8fe8160480 2024-05-11T01:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.940+0000 7f8fee504640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe8163530 con 0x7f8fe815f430 2024-05-11T01:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.940+0000 7f8fee504640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe8163810 con 0x7f8fe80b5dd0 2024-05-11T01:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.940+0000 7f8fe77fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8fe8160480 0x7f8fe80d2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58164/0 (socket says 172.21.15.52:58164) 2024-05-11T01:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.940+0000 7f8fe77fe640 1 -- 172.21.15.52:0/3938757432 learned_addr learned my addr 172.21.15.52:0/3938757432 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.940+0000 7f8fecaa1640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80b5dd0 0x7f8fe80d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.940+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887412236 0 0) 0x7f8fe8163250 con 0x7f8fe8160480 2024-05-11T01:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.940+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fd0003520 con 0x7f8fe8160480 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1737769180 0 0) 0x7f8fe8163530 con 0x7f8fe815f430 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fe8163250 con 0x7f8fe815f430 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1866162483 0 0) 0x7f8fd0003520 con 0x7f8fe8160480 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80b5dd0 msgr2=0x7f8fe80d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80b5dd0 0x7f8fe80d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 >> v1:172.21.15.119:6789/0 conn(0x7f8fe815f430 legacy=0x7f8fe815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe8163af0 con 0x7f8fe8160480 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f8fdc001710 con 0x7f8fe8160480 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fecaa1640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80b5dd0 0x7f8fe80d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 >> v1:172.21.15.52:6789/0 conn(0x7f8fe8160480 legacy=0x7f8fe80d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fd0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0004800 0x7f8fd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80b5dd0 0x7f8fd00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8163810 con 0x7f8fd0003c10 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd000bad0 con 0x7f8fd0004800 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd000bcd0 con 0x7f8fe80b5dd0 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe7fff640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fd0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe77fe640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0004800 0x7f8fd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fecaa1640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80b5dd0 0x7f8fd00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe7fff640 1 -- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80b5dd0 msgr2=0x7f8fd00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe7fff640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80b5dd0 0x7f8fd00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe7fff640 1 -- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0004800 msgr2=0x7f8fd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe7fff640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0004800 0x7f8fd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.941+0000 7f8fe7fff640 1 -- 172.21.15.52:0/3938757432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd0003520 con 0x7f8fd0003c10 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.942+0000 7f8fe7fff640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fd0004010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8001900 tx=0x7f8fd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd272c3fb8fa59b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.942+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8001520 con 0x7f8fd0003c10 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.942+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd802acd0 con 0x7f8fd0003c10 2024-05-11T01:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.942+0000 7f8fe6ffd640 1 -- 172.21.15.52:0/3938757432 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8005310 con 0x7f8fd0003c10 2024-05-11T01:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.943+0000 7f8fee504640 1 -- 172.21.15.52:0/3938757432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 msgr2=0x7f8fd0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.943+0000 7f8fee504640 1 --2- 172.21.15.52:0/3938757432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fd0004010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8001900 tx=0x7f8fd0005830 comp rx=0 tx=0).stop 2024-05-11T01:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.943+0000 7f8fecaa1640 1 -- 172.21.15.52:0/3938757432 reap_dead start 2024-05-11T01:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.943+0000 7f8fee504640 1 -- 172.21.15.52:0/3938757432 shutdown_connections 2024-05-11T01:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.943+0000 7f8fee504640 1 -- 172.21.15.52:0/3938757432 >> 172.21.15.52:0/3938757432 conn(0x7f8fe808eb70 msgr2=0x7f8fe809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.943+0000 7f8fee504640 1 -- 172.21.15.52:0/3938757432 shutdown_connections 2024-05-11T01:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.943+0000 7f8fee504640 1 -- 172.21.15.52:0/3938757432 wait complete. 2024-05-11T01:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.944+0000 7f8fee504640 1 Processor -- start 2024-05-11T01:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.944+0000 7f8fee504640 1 -- start start 2024-05-11T01:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fee504640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fe81f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fee504640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80cc3a0 0x7f8fe815aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fee504640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe8153850 0x7f8fe8162320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fee504640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8163030 con 0x7f8fd0003c10 2024-05-11T01:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fee504640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8162eb0 con 0x7f8fe8153850 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fee504640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe81608f0 con 0x7f8fe80cc3a0 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fe81f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fe81f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35396/0 (socket says 172.21.15.52:35396) 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe7fff640 1 -- 172.21.15.52:0/2978941674 learned_addr learned my addr 172.21.15.52:0/2978941674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe77fe640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80cc3a0 0x7f8fe815aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fecaa1640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe8153850 0x7f8fe8162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe7fff640 1 -- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80cc3a0 msgr2=0x7f8fe815aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe7fff640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80cc3a0 0x7f8fe815aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe7fff640 1 -- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe8153850 msgr2=0x7f8fe8162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe7fff640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe8153850 0x7f8fe8162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe7fff640 1 -- 172.21.15.52:0/2978941674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe815b420 con 0x7f8fd0003c10 2024-05-11T01:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.945+0000 7f8fe7fff640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fe81f85b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005040 tx=0x7f8fd80012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.946+0000 7f8fe4ff9640 1 -- 172.21.15.52:0/2978941674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8001520 con 0x7f8fd0003c10 2024-05-11T01:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.946+0000 7f8fe4ff9640 1 -- 172.21.15.52:0/2978941674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd80075e0 con 0x7f8fd0003c10 2024-05-11T01:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.946+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe80cdce0 con 0x7f8fd0003c10 2024-05-11T01:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.946+0000 7f8fe4ff9640 1 -- 172.21.15.52:0/2978941674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8007a10 con 0x7f8fd0003c10 2024-05-11T01:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.946+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81f8fe0 con 0x7f8fd0003c10 2024-05-11T01:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.947+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe8069780 con 0x7f8fd0003c10 2024-05-11T01:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.947+0000 7f8fe4ff9640 1 -- 172.21.15.52:0/2978941674 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8fd8044060 con 0x7f8fd0003c10 2024-05-11T01:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.948+0000 7f8fe4ff9640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f8fd40762d0 0x7f8fd4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.948+0000 7f8fe4ff9640 1 -- 172.21.15.52:0/2978941674 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8fd80b6750 con 0x7f8fd0003c10 2024-05-11T01:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.948+0000 7f8fe77fe640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f8fd40762d0 0x7f8fd4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.948+0000 7f8fe77fe640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f8fd40762d0 0x7f8fd4078790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc001050 tx=0x7f8fdc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:14.952+0000 7f8fe4ff9640 1 -- 172.21.15.52:0/2978941674 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8fe80cdce0 con 0x7f8fd0003c10 2024-05-11T01:01:15.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.139+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fe80b4ba0 con 0x7f8fd40762d0 2024-05-11T01:01:15.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.146+0000 7f8fe4ff9640 1 -- 172.21.15.52:0/2978941674 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f8fe80b4ba0 con 0x7f8fd40762d0 2024-05-11T01:01:15.150 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:15.151 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:01:15.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f8fd40762d0 msgr2=0x7f8fd4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f8fd40762d0 0x7f8fd4078790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc001050 tx=0x7f8fdc009040 comp rx=0 tx=0).stop 2024-05-11T01:01:15.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 msgr2=0x7f8fe81f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fe81f85b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005040 tx=0x7f8fd80012f0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 shutdown_connections 2024-05-11T01:01:15.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f8fd40762d0 0x7f8fd4078790 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fe8153850 0x7f8fe8162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fe80cc3a0 0x7f8fe815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 --2- 172.21.15.52:0/2978941674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fd0003c10 0x7f8fe81f85b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 >> 172.21.15.52:0/2978941674 conn(0x7f8fe808eb70 msgr2=0x7f8fe80d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:15.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 shutdown_connections 2024-05-11T01:01:15.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.148+0000 7f8fee504640 1 -- 172.21.15.52:0/2978941674 wait complete. 2024-05-11T01:01:15.161 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-05-11T01:01:13.513017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235632,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739088,"statfs":{"total":773094113280,"available":772852826112,"internally_reserved":0,"allocated":2891776,"data_stored":1750788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.837260"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794175+0000","last_change":"2024-05-11T01:01:01.690368+0000","last_active":"2024-05-11T01:01:02.794175+0000","last_peered":"2024-05-11T01:01:02.794175+0000","last_clean":"2024-05-11T01:01:02.794175+0000","last_became_active":"2024-05-11T01:01:01.689860+0000","last_became_peered":"2024-05-11T01:01:01.689860+0000","last_unstale":"2024-05-11T01:01:02.794175+0000","last_undegraded":"2024-05-11T01:01:02.794175+0000","last_fullsized":"2024-05-11T01:01:02.794175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:22:27.780116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703164+0000","last_change":"2024-05-11T01:01:01.692121+0000","last_active":"2024-05-11T01:01:02.703164+0000","last_peered":"2024-05-11T01:01:02.703164+0000","last_clean":"2024-05-11T01:01:02.703164+0000","last_became_active":"2024-05-11T01:01:01.691843+0000","last_became_peered":"2024-05-11T01:01:01.691843+0000","last_unstale":"2024-05-11T01:01:02.703164+0000","last_undegraded":"2024-05-11T01:01:02.703164+0000","last_fullsized":"2024-05-11T01:01:02.703164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:04:19.015364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676930+0000","last_change":"2024-05-11T01:01:01.681630+0000","last_active":"2024-05-11T01:01:02.676930+0000","last_peered":"2024-05-11T01:01:02.676930+0000","last_clean":"2024-05-11T01:01:02.676930+0000","last_became_active":"2024-05-11T01:01:01.681342+0000","last_became_peered":"2024-05-11T01:01:01.681342+0000","last_unstale":"2024-05-11T01:01:02.676930+0000","last_undegraded":"2024-05-11T01:01:02.676930+0000","last_fullsized":"2024-05-11T01:01:02.676930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:02:55.828068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678747+0000","last_change":"2024-05-11T01:01:01.687729+0000","last_active":"2024-05-11T01:01:02.678747+0000","last_peered":"2024-05-11T01:01:02.678747+0000","last_clean":"2024-05-11T01:01:02.678747+0000","last_became_active":"2024-05-11T01:01:01.687234+0000","last_became_peered":"2024-05-11T01:01:01.687234+0000","last_unstale":"2024-05-11T01:01:02.678747+0000","last_undegraded":"2024-05-11T01:01:02.678747+0000","last_fullsized":"2024-05-11T01:01:02.678747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:10.357158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793604+0000","last_change":"2024-05-11T01:01:01.687598+0000","last_active":"2024-05-11T01:01:02.793604+0000","last_peered":"2024-05-11T01:01:02.793604+0000","last_clean":"2024-05-11T01:01:02.793604+0000","last_became_active":"2024-05-11T01:01:01.687326+0000","last_became_peered":"2024-05-11T01:01:01.687326+0000","last_unstale":"2024-05-11T01:01:02.793604+0000","last_undegraded":"2024-05-11T01:01:02.793604+0000","last_fullsized":"2024-05-11T01:01:02.793604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677883+0000","last_change":"2024-05-11T01:01:01.688617+0000","last_active":"2024-05-11T01:01:02.677883+0000","last_peered":"2024-05-11T01:01:02.677883+0000","last_clean":"2024-05-11T01:01:02.677883+0000","last_became_active":"2024-05-11T01:01:01.687985+0000","last_became_peered":"2024-05-11T01:01:01.687985+0000","last_unstale":"2024-05-11T01:01:02.677883+0000","last_undegraded":"2024-05-11T01:01:02.677883+0000","last_fullsized":"2024-05-11T01:01:02.677883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:24.445230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793490+0000","last_change":"2024-05-11T01:01:01.692158+0000","last_active":"2024-05-11T01:01:02.793490+0000","last_peered":"2024-05-11T01:01:02.793490+0000","last_clean":"2024-05-11T01:01:02.793490+0000","last_became_active":"2024-05-11T01:01:01.691929+0000","last_became_peered":"2024-05-11T01:01:01.691929+0000","last_unstale":"2024-05-11T01:01:02.793490+0000","last_undegraded":"2024-05-11T01:01:02.793490+0000","last_fullsized":"2024-05-11T01:01:02.793490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:24.858093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912954+0000","last_change":"2024-05-11T01:01:01.694128+0000","last_active":"2024-05-11T01:01:02.912954+0000","last_peered":"2024-05-11T01:01:02.912954+0000","last_clean":"2024-05-11T01:01:02.912954+0000","last_became_active":"2024-05-11T01:01:01.693513+0000","last_became_peered":"2024-05-11T01:01:01.693513+0000","last_unstale":"2024-05-11T01:01:02.912954+0000","last_undegraded":"2024-05-11T01:01:02.912954+0000","last_fullsized":"2024-05-11T01:01:02.912954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:02:55.828068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794149+0000","last_change":"2024-05-11T01:01:01.705504+0000","last_active":"2024-05-11T01:01:02.794149+0000","last_peered":"2024-05-11T01:01:02.794149+0000","last_clean":"2024-05-11T01:01:02.794149+0000","last_became_active":"2024-05-11T01:01:01.705258+0000","last_became_peered":"2024-05-11T01:01:01.705258+0000","last_unstale":"2024-05-11T01:01:02.794149+0000","last_undegraded":"2024-05-11T01:01:02.794149+0000","last_fullsized":"2024-05-11T01:01:02.794149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:00:03.997273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705744+0000","last_change":"2024-05-11T01:01:01.696662+0000","last_active":"2024-05-11T01:01:02.705744+0000","last_peered":"2024-05-11T01:01:02.705744+0000","last_clean":"2024-05-11T01:01:02.705744+0000","last_became_active":"2024-05-11T01:01:01.696364+0000","last_became_peered":"2024-05-11T01:01:01.696364+0000","last_unstale":"2024-05-11T01:01:02.705744+0000","last_undegraded":"2024-05-11T01:01:02.705744+0000","last_fullsized":"2024-05-11T01:01:02.705744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:04:19.015364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705571+0000","last_change":"2024-05-11T01:01:01.699790+0000","last_active":"2024-05-11T01:01:02.705571+0000","last_peered":"2024-05-11T01:01:02.705571+0000","last_clean":"2024-05-11T01:01:02.705571+0000","last_became_active":"2024-05-11T01:01:01.699494+0000","last_became_peered":"2024-05-11T01:01:01.699494+0000","last_unstale":"2024-05-11T01:01:02.705571+0000","last_undegraded":"2024-05-11T01:01:02.705571+0000","last_fullsized":"2024-05-11T01:01:02.705571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678303+0000","last_change":"2024-05-11T01:01:01.696604+0000","last_active":"2024-05-11T01:01:02.678303+0000","last_peered":"2024-05-11T01:01:02.678303+0000","last_clean":"2024-05-11T01:01:02.678303+0000","last_became_active":"2024-05-11T01:01:01.695857+0000","last_became_peered":"2024-05-11T01:01:01.695857+0000","last_unstale":"2024-05-11T01:01:02.678303+0000","last_undegraded":"2024-05-11T01:01:02.678303+0000","last_fullsized":"2024-05-11T01:01:02.678303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:06.977149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913350+0000","last_change":"2024-05-11T01:01:01.690950+0000","last_active":"2024-05-11T01:01:02.913350+0000","last_peered":"2024-05-11T01:01:02.913350+0000","last_clean":"2024-05-11T01:01:02.913350+0000","last_became_active":"2024-05-11T01:01:01.690683+0000","last_became_peered":"2024-05-11T01:01:01.690683+0000","last_unstale":"2024-05-11T01:01:02.913350+0000","last_undegraded":"2024-05-11T01:01:02.913350+0000","last_fullsized":"2024-05-11T01:01:02.913350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676647+0000","last_change":"2024-05-11T01:01:01.690525+0000","last_active":"2024-05-11T01:01:02.676647+0000","last_peered":"2024-05-11T01:01:02.676647+0000","last_clean":"2024-05-11T01:01:02.676647+0000","last_became_active":"2024-05-11T01:01:01.690246+0000","last_became_peered":"2024-05-11T01:01:01.690246+0000","last_unstale":"2024-05-11T01:01:02.676647+0000","last_undegraded":"2024-05-11T01:01:02.676647+0000","last_fullsized":"2024-05-11T01:01:02.676647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:59.768543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.702977+0000","last_change":"2024-05-11T01:01:01.696660+0000","last_active":"2024-05-11T01:01:02.702977+0000","last_peered":"2024-05-11T01:01:02.702977+0000","last_clean":"2024-05-11T01:01:02.702977+0000","last_became_active":"2024-05-11T01:01:01.696344+0000","last_became_peered":"2024-05-11T01:01:01.696344+0000","last_unstale":"2024-05-11T01:01:02.702977+0000","last_undegraded":"2024-05-11T01:01:02.702977+0000","last_fullsized":"2024-05-11T01:01:02.702977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677628+0000","last_change":"2024-05-11T01:01:01.691118+0000","last_active":"2024-05-11T01:01:02.677628+0000","last_peered":"2024-05-11T01:01:02.677628+0000","last_clean":"2024-05-11T01:01:02.677628+0000","last_became_active":"2024-05-11T01:01:01.690850+0000","last_became_peered":"2024-05-11T01:01:01.690850+0000","last_unstale":"2024-05-11T01:01:02.677628+0000","last_undegraded":"2024-05-11T01:01:02.677628+0000","last_fullsized":"2024-05-11T01:01:02.677628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678640+0000","last_change":"2024-05-11T01:01:01.691696+0000","last_active":"2024-05-11T01:01:02.678640+0000","last_peered":"2024-05-11T01:01:02.678640+0000","last_clean":"2024-05-11T01:01:02.678640+0000","last_became_active":"2024-05-11T01:01:01.691424+0000","last_became_peered":"2024-05-11T01:01:01.691424+0000","last_unstale":"2024-05-11T01:01:02.678640+0000","last_undegraded":"2024-05-11T01:01:02.678640+0000","last_fullsized":"2024-05-11T01:01:02.678640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:13:49.063574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913015+0000","last_change":"2024-05-11T01:01:01.690778+0000","last_active":"2024-05-11T01:01:02.913015+0000","last_peered":"2024-05-11T01:01:02.913015+0000","last_clean":"2024-05-11T01:01:02.913015+0000","last_became_active":"2024-05-11T01:01:01.690403+0000","last_became_peered":"2024-05-11T01:01:01.690403+0000","last_unstale":"2024-05-11T01:01:02.913015+0000","last_undegraded":"2024-05-11T01:01:02.913015+0000","last_fullsized":"2024-05-11T01:01:02.913015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:59.768543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794101+0000","last_change":"2024-05-11T01:01:01.697511+0000","last_active":"2024-05-11T01:01:02.794101+0000","last_peered":"2024-05-11T01:01:02.794101+0000","last_clean":"2024-05-11T01:01:02.794101+0000","last_became_active":"2024-05-11T01:01:01.697263+0000","last_became_peered":"2024-05-11T01:01:01.697263+0000","last_unstale":"2024-05-11T01:01:02.794101+0000","last_undegraded":"2024-05-11T01:01:02.794101+0000","last_fullsized":"2024-05-11T01:01:02.794101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:10.357158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705536+0000","last_change":"2024-05-11T01:01:01.697194+0000","last_active":"2024-05-11T01:01:02.705536+0000","last_peered":"2024-05-11T01:01:02.705536+0000","last_clean":"2024-05-11T01:01:02.705536+0000","last_became_active":"2024-05-11T01:01:01.696909+0000","last_became_peered":"2024-05-11T01:01:01.696909+0000","last_unstale":"2024-05-11T01:01:02.705536+0000","last_undegraded":"2024-05-11T01:01:02.705536+0000","last_fullsized":"2024-05-11T01:01:02.705536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676985+0000","last_change":"2024-05-11T01:01:01.690623+0000","last_active":"2024-05-11T01:01:02.676985+0000","last_peered":"2024-05-11T01:01:02.676985+0000","last_clean":"2024-05-11T01:01:02.676985+0000","last_became_active":"2024-05-11T01:01:01.689762+0000","last_became_peered":"2024-05-11T01:01:01.689762+0000","last_unstale":"2024-05-11T01:01:02.676985+0000","last_undegraded":"2024-05-11T01:01:02.676985+0000","last_fullsized":"2024-05-11T01:01:02.676985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:10:39.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678044+0000","last_change":"2024-05-11T01:01:01.692409+0000","last_active":"2024-05-11T01:01:02.678044+0000","last_peered":"2024-05-11T01:01:02.678044+0000","last_clean":"2024-05-11T01:01:02.678044+0000","last_became_active":"2024-05-11T01:01:01.692117+0000","last_became_peered":"2024-05-11T01:01:01.692117+0000","last_unstale":"2024-05-11T01:01:02.678044+0000","last_undegraded":"2024-05-11T01:01:02.678044+0000","last_fullsized":"2024-05-11T01:01:02.678044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:36:33.062657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913338+0000","last_change":"2024-05-11T01:01:01.697812+0000","last_active":"2024-05-11T01:01:02.913338+0000","last_peered":"2024-05-11T01:01:02.913338+0000","last_clean":"2024-05-11T01:01:02.913338+0000","last_became_active":"2024-05-11T01:01:01.697509+0000","last_became_peered":"2024-05-11T01:01:01.697509+0000","last_unstale":"2024-05-11T01:01:02.913338+0000","last_undegraded":"2024-05-11T01:01:02.913338+0000","last_fullsized":"2024-05-11T01:01:02.913338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678573+0000","last_change":"2024-05-11T01:01:01.689599+0000","last_active":"2024-05-11T01:01:02.678573+0000","last_peered":"2024-05-11T01:01:02.678573+0000","last_clean":"2024-05-11T01:01:02.678573+0000","last_became_active":"2024-05-11T01:01:01.686994+0000","last_became_peered":"2024-05-11T01:01:01.686994+0000","last_unstale":"2024-05-11T01:01:02.678573+0000","last_undegraded":"2024-05-11T01:01:02.678573+0000","last_fullsized":"2024-05-11T01:01:02.678573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:32.857796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913719+0000","last_change":"2024-05-11T01:01:01.699000+0000","last_active":"2024-05-11T01:01:02.913719+0000","last_peered":"2024-05-11T01:01:02.913719+0000","last_clean":"2024-05-11T01:01:02.913719+0000","last_became_active":"2024-05-11T01:01:01.698640+0000","last_became_peered":"2024-05-11T01:01:01.698640+0000","last_unstale":"2024-05-11T01:01:02.913719+0000","last_undegraded":"2024-05-11T01:01:02.913719+0000","last_fullsized":"2024-05-11T01:01:02.913719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:24.445230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705756+0000","last_change":"2024-05-11T01:01:01.689683+0000","last_active":"2024-05-11T01:01:02.705756+0000","last_peered":"2024-05-11T01:01:02.705756+0000","last_clean":"2024-05-11T01:01:02.705756+0000","last_became_active":"2024-05-11T01:01:01.689306+0000","last_became_peered":"2024-05-11T01:01:01.689306+0000","last_unstale":"2024-05-11T01:01:02.705756+0000","last_undegraded":"2024-05-11T01:01:02.705756+0000","last_fullsized":"2024-05-11T01:01:02.705756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703176+0000","last_change":"2024-05-11T01:01:01.694512+0000","last_active":"2024-05-11T01:01:02.703176+0000","last_peered":"2024-05-11T01:01:02.703176+0000","last_clean":"2024-05-11T01:01:02.703176+0000","last_became_active":"2024-05-11T01:01:01.694214+0000","last_became_peered":"2024-05-11T01:01:01.694214+0000","last_unstale":"2024-05-11T01:01:02.703176+0000","last_undegraded":"2024-05-11T01:01:02.703176+0000","last_fullsized":"2024-05-11T01:01:02.703176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913695+0000","last_change":"2024-05-11T01:01:01.698983+0000","last_active":"2024-05-11T01:01:02.913695+0000","last_peered":"2024-05-11T01:01:02.913695+0000","last_clean":"2024-05-11T01:01:02.913695+0000","last_became_active":"2024-05-11T01:01:01.698682+0000","last_became_peered":"2024-05-11T01:01:01.698682+0000","last_unstale":"2024-05-11T01:01:02.913695+0000","last_undegraded":"2024-05-11T01:01:02.913695+0000","last_fullsized":"2024-05-11T01:01:02.913695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:04:19.015364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676868+0000","last_change":"2024-05-11T01:01:01.690633+0000","last_active":"2024-05-11T01:01:02.676868+0000","last_peered":"2024-05-11T01:01:02.676868+0000","last_clean":"2024-05-11T01:01:02.676868+0000","last_became_active":"2024-05-11T01:01:01.690227+0000","last_became_peered":"2024-05-11T01:01:01.690227+0000","last_unstale":"2024-05-11T01:01:02.676868+0000","last_undegraded":"2024-05-11T01:01:02.676868+0000","last_fullsized":"2024-05-11T01:01:02.676868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:24.703345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678131+0000","last_change":"2024-05-11T01:01:01.688827+0000","last_active":"2024-05-11T01:01:02.678131+0000","last_peered":"2024-05-11T01:01:02.678131+0000","last_clean":"2024-05-11T01:01:02.678131+0000","last_became_active":"2024-05-11T01:01:01.688376+0000","last_became_peered":"2024-05-11T01:01:01.688376+0000","last_unstale":"2024-05-11T01:01:02.678131+0000","last_undegraded":"2024-05-11T01:01:02.678131+0000","last_fullsized":"2024-05-11T01:01:02.678131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:26:24.858093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912912+0000","last_change":"2024-05-11T01:01:01.692633+0000","last_active":"2024-05-11T01:01:02.912912+0000","last_peered":"2024-05-11T01:01:02.912912+0000","last_clean":"2024-05-11T01:01:02.912912+0000","last_became_active":"2024-05-11T01:01:01.692305+0000","last_became_peered":"2024-05-11T01:01:01.692305+0000","last_unstale":"2024-05-11T01:01:02.912912+0000","last_undegraded":"2024-05-11T01:01:02.912912+0000","last_fullsized":"2024-05-11T01:01:02.912912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:10:34.198396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705339+0000","last_change":"2024-05-11T01:01:01.692423+0000","last_active":"2024-05-11T01:01:02.705339+0000","last_peered":"2024-05-11T01:01:02.705339+0000","last_clean":"2024-05-11T01:01:02.705339+0000","last_became_active":"2024-05-11T01:01:01.691669+0000","last_became_peered":"2024-05-11T01:01:01.691669+0000","last_unstale":"2024-05-11T01:01:02.705339+0000","last_undegraded":"2024-05-11T01:01:02.705339+0000","last_fullsized":"2024-05-11T01:01:02.705339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:10:39.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677711+0000","last_change":"2024-05-11T01:01:01.691243+0000","last_active":"2024-05-11T01:01:02.677711+0000","last_peered":"2024-05-11T01:01:02.677711+0000","last_clean":"2024-05-11T01:01:02.677711+0000","last_became_active":"2024-05-11T01:01:01.690974+0000","last_became_peered":"2024-05-11T01:01:01.690974+0000","last_unstale":"2024-05-11T01:01:02.677711+0000","last_undegraded":"2024-05-11T01:01:02.677711+0000","last_fullsized":"2024-05-11T01:01:02.677711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:25.558002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913441+0000","last_change":"2024-05-11T01:01:00.681960+0000","last_active":"2024-05-11T01:01:02.913441+0000","last_peered":"2024-05-11T01:01:02.913441+0000","last_clean":"2024-05-11T01:01:02.913441+0000","last_became_active":"2024-05-11T01:01:00.681658+0000","last_became_peered":"2024-05-11T01:01:00.681658+0000","last_unstale":"2024-05-11T01:01:02.913441+0000","last_undegraded":"2024-05-11T01:01:02.913441+0000","last_fullsized":"2024-05-11T01:01:02.913441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676953+0000","last_change":"2024-05-11T01:01:01.697201+0000","last_active":"2024-05-11T01:01:02.676953+0000","last_peered":"2024-05-11T01:01:02.676953+0000","last_clean":"2024-05-11T01:01:02.676953+0000","last_became_active":"2024-05-11T01:01:01.696525+0000","last_became_peered":"2024-05-11T01:01:01.696525+0000","last_unstale":"2024-05-11T01:01:02.676953+0000","last_undegraded":"2024-05-11T01:01:02.676953+0000","last_fullsized":"2024-05-11T01:01:02.676953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:27:35.463648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705370+0000","last_change":"2024-05-11T01:01:00.684578+0000","last_active":"2024-05-11T01:01:02.705370+0000","last_peered":"2024-05-11T01:01:02.705370+0000","last_clean":"2024-05-11T01:01:02.705370+0000","last_became_active":"2024-05-11T01:01:00.684303+0000","last_became_peered":"2024-05-11T01:01:00.684303+0000","last_unstale":"2024-05-11T01:01:02.705370+0000","last_undegraded":"2024-05-11T01:01:02.705370+0000","last_fullsized":"2024-05-11T01:01:02.705370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913278+0000","last_change":"2024-05-11T01:01:00.678144+0000","last_active":"2024-05-11T01:01:02.913278+0000","last_peered":"2024-05-11T01:01:02.913278+0000","last_clean":"2024-05-11T01:01:02.913278+0000","last_became_active":"2024-05-11T01:01:00.677790+0000","last_became_peered":"2024-05-11T01:01:00.677790+0000","last_unstale":"2024-05-11T01:01:02.913278+0000","last_undegraded":"2024-05-11T01:01:02.913278+0000","last_fullsized":"2024-05-11T01:01:02.913278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:38:37.156429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913068+0000","last_change":"2024-05-11T01:01:01.689344+0000","last_active":"2024-05-11T01:01:02.913068+0000","last_peered":"2024-05-11T01:01:02.913068+0000","last_clean":"2024-05-11T01:01:02.913068+0000","last_became_active":"2024-05-11T01:01:01.687984+0000","last_became_peered":"2024-05-11T01:01:01.687984+0000","last_unstale":"2024-05-11T01:01:02.913068+0000","last_undegraded":"2024-05-11T01:01:02.913068+0000","last_fullsized":"2024-05-11T01:01:02.913068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:27:35.463648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678187+0000","last_change":"2024-05-11T01:01:00.678981+0000","last_active":"2024-05-11T01:01:02.678187+0000","last_peered":"2024-05-11T01:01:02.678187+0000","last_clean":"2024-05-11T01:01:02.678187+0000","last_became_active":"2024-05-11T01:01:00.678672+0000","last_became_peered":"2024-05-11T01:01:00.678672+0000","last_unstale":"2024-05-11T01:01:02.678187+0000","last_undegraded":"2024-05-11T01:01:02.678187+0000","last_fullsized":"2024-05-11T01:01:02.678187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.702964+0000","last_change":"2024-05-11T01:01:01.694506+0000","last_active":"2024-05-11T01:01:02.702964+0000","last_peered":"2024-05-11T01:01:02.702964+0000","last_clean":"2024-05-11T01:01:02.702964+0000","last_became_active":"2024-05-11T01:01:01.694226+0000","last_became_peered":"2024-05-11T01:01:01.694226+0000","last_unstale":"2024-05-11T01:01:02.702964+0000","last_undegraded":"2024-05-11T01:01:02.702964+0000","last_fullsized":"2024-05-11T01:01:02.702964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705643+0000","last_change":"2024-05-11T01:01:00.679681+0000","last_active":"2024-05-11T01:01:02.705643+0000","last_peered":"2024-05-11T01:01:02.705643+0000","last_clean":"2024-05-11T01:01:02.705643+0000","last_became_active":"2024-05-11T01:01:00.679125+0000","last_became_peered":"2024-05-11T01:01:00.679125+0000","last_unstale":"2024-05-11T01:01:02.705643+0000","last_undegraded":"2024-05-11T01:01:02.705643+0000","last_fullsized":"2024-05-11T01:01:02.705643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678282+0000","last_change":"2024-05-11T01:01:01.690424+0000","last_active":"2024-05-11T01:01:02.678282+0000","last_peered":"2024-05-11T01:01:02.678282+0000","last_clean":"2024-05-11T01:01:02.678282+0000","last_became_active":"2024-05-11T01:01:01.690131+0000","last_became_peered":"2024-05-11T01:01:01.690131+0000","last_unstale":"2024-05-11T01:01:02.678282+0000","last_undegraded":"2024-05-11T01:01:02.678282+0000","last_fullsized":"2024-05-11T01:01:02.678282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:35:54.225824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678389+0000","last_change":"2024-05-11T01:01:00.676347+0000","last_active":"2024-05-11T01:01:02.678389+0000","last_peered":"2024-05-11T01:01:02.678389+0000","last_clean":"2024-05-11T01:01:02.678389+0000","last_became_active":"2024-05-11T01:01:00.676058+0000","last_became_peered":"2024-05-11T01:01:00.676058+0000","last_unstale":"2024-05-11T01:01:02.678389+0000","last_undegraded":"2024-05-11T01:01:02.678389+0000","last_fullsized":"2024-05-11T01:01:02.678389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677252+0000","last_change":"2024-05-11T01:01:01.685176+0000","last_active":"2024-05-11T01:01:02.677252+0000","last_peered":"2024-05-11T01:01:02.677252+0000","last_clean":"2024-05-11T01:01:02.677252+0000","last_became_active":"2024-05-11T01:01:01.684804+0000","last_became_peered":"2024-05-11T01:01:01.684804+0000","last_unstale":"2024-05-11T01:01:02.677252+0000","last_undegraded":"2024-05-11T01:01:02.677252+0000","last_fullsized":"2024-05-11T01:01:02.677252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703190+0000","last_change":"2024-05-11T01:01:00.673845+0000","last_active":"2024-05-11T01:01:02.703190+0000","last_peered":"2024-05-11T01:01:02.703190+0000","last_clean":"2024-05-11T01:01:02.703190+0000","last_became_active":"2024-05-11T01:01:00.673430+0000","last_became_peered":"2024-05-11T01:01:00.673430+0000","last_unstale":"2024-05-11T01:01:02.703190+0000","last_undegraded":"2024-05-11T01:01:02.703190+0000","last_fullsized":"2024-05-11T01:01:02.703190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678463+0000","last_change":"2024-05-11T01:01:01.695552+0000","last_active":"2024-05-11T01:01:02.678463+0000","last_peered":"2024-05-11T01:01:02.678463+0000","last_clean":"2024-05-11T01:01:02.678463+0000","last_became_active":"2024-05-11T01:01:01.693674+0000","last_became_peered":"2024-05-11T01:01:01.693674+0000","last_unstale":"2024-05-11T01:01:02.678463+0000","last_undegraded":"2024-05-11T01:01:02.678463+0000","last_fullsized":"2024-05-11T01:01:02.678463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:17.700359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705505+0000","last_change":"2024-05-11T01:01:00.683575+0000","last_active":"2024-05-11T01:01:02.705505+0000","last_peered":"2024-05-11T01:01:02.705505+0000","last_clean":"2024-05-11T01:01:02.705505+0000","last_became_active":"2024-05-11T01:01:00.683259+0000","last_became_peered":"2024-05-11T01:01:00.683259+0000","last_unstale":"2024-05-11T01:01:02.705505+0000","last_undegraded":"2024-05-11T01:01:02.705505+0000","last_fullsized":"2024-05-11T01:01:02.705505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793910+0000","last_change":"2024-05-11T01:01:01.699906+0000","last_active":"2024-05-11T01:01:02.793910+0000","last_peered":"2024-05-11T01:01:02.793910+0000","last_clean":"2024-05-11T01:01:02.793910+0000","last_became_active":"2024-05-11T01:01:01.699637+0000","last_became_peered":"2024-05-11T01:01:01.699637+0000","last_unstale":"2024-05-11T01:01:02.793910+0000","last_undegraded":"2024-05-11T01:01:02.793910+0000","last_fullsized":"2024-05-11T01:01:02.793910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:13:49.063574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677033+0000","last_change":"2024-05-11T01:01:00.684668+0000","last_active":"2024-05-11T01:01:02.677033+0000","last_peered":"2024-05-11T01:01:02.677033+0000","last_clean":"2024-05-11T01:01:02.677033+0000","last_became_active":"2024-05-11T01:01:00.684364+0000","last_became_peered":"2024-05-11T01:01:00.684364+0000","last_unstale":"2024-05-11T01:01:02.677033+0000","last_undegraded":"2024-05-11T01:01:02.677033+0000","last_fullsized":"2024-05-11T01:01:02.677033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:54.494836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676757+0000","last_change":"2024-05-11T01:01:01.686608+0000","last_active":"2024-05-11T01:01:02.676757+0000","last_peered":"2024-05-11T01:01:02.676757+0000","last_clean":"2024-05-11T01:01:02.676757+0000","last_became_active":"2024-05-11T01:01:01.686333+0000","last_became_peered":"2024-05-11T01:01:01.686333+0000","last_unstale":"2024-05-11T01:01:02.676757+0000","last_undegraded":"2024-05-11T01:01:02.676757+0000","last_fullsized":"2024-05-11T01:01:02.676757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:51.874581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678282+0000","last_change":"2024-05-11T01:01:00.684592+0000","last_active":"2024-05-11T01:01:02.678282+0000","last_peered":"2024-05-11T01:01:02.678282+0000","last_clean":"2024-05-11T01:01:02.678282+0000","last_became_active":"2024-05-11T01:01:00.683709+0000","last_became_peered":"2024-05-11T01:01:00.683709+0000","last_unstale":"2024-05-11T01:01:02.678282+0000","last_undegraded":"2024-05-11T01:01:02.678282+0000","last_fullsized":"2024-05-11T01:01:02.678282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793773+0000","last_change":"2024-05-11T01:01:01.699198+0000","last_active":"2024-05-11T01:01:02.793773+0000","last_peered":"2024-05-11T01:01:02.793773+0000","last_clean":"2024-05-11T01:01:02.793773+0000","last_became_active":"2024-05-11T01:01:01.698959+0000","last_became_peered":"2024-05-11T01:01:01.698959+0000","last_unstale":"2024-05-11T01:01:02.793773+0000","last_undegraded":"2024-05-11T01:01:02.793773+0000","last_fullsized":"2024-05-11T01:01:02.793773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:32.857796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705344+0000","last_change":"2024-05-11T01:01:00.684714+0000","last_active":"2024-05-11T01:01:02.705344+0000","last_peered":"2024-05-11T01:01:02.705344+0000","last_clean":"2024-05-11T01:01:02.705344+0000","last_became_active":"2024-05-11T01:01:00.684454+0000","last_became_peered":"2024-05-11T01:01:00.684454+0000","last_unstale":"2024-05-11T01:01:02.705344+0000","last_undegraded":"2024-05-11T01:01:02.705344+0000","last_fullsized":"2024-05-11T01:01:02.705344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705449+0000","last_change":"2024-05-11T01:01:01.691666+0000","last_active":"2024-05-11T01:01:02.705449+0000","last_peered":"2024-05-11T01:01:02.705449+0000","last_clean":"2024-05-11T01:01:02.705449+0000","last_became_active":"2024-05-11T01:01:01.691368+0000","last_became_peered":"2024-05-11T01:01:01.691368+0000","last_unstale":"2024-05-11T01:01:02.705449+0000","last_undegraded":"2024-05-11T01:01:02.705449+0000","last_fullsized":"2024-05-11T01:01:02.705449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912707+0000","last_change":"2024-05-11T01:01:00.680506+0000","last_active":"2024-05-11T01:01:02.912707+0000","last_peered":"2024-05-11T01:01:02.912707+0000","last_clean":"2024-05-11T01:01:02.912707+0000","last_became_active":"2024-05-11T01:01:00.680198+0000","last_became_peered":"2024-05-11T01:01:00.680198+0000","last_unstale":"2024-05-11T01:01:02.912707+0000","last_undegraded":"2024-05-11T01:01:02.912707+0000","last_fullsized":"2024-05-11T01:01:02.912707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793544+0000","last_change":"2024-05-11T01:01:01.698747+0000","last_active":"2024-05-11T01:01:02.793544+0000","last_peered":"2024-05-11T01:01:02.793544+0000","last_clean":"2024-05-11T01:01:02.793544+0000","last_became_active":"2024-05-11T01:01:01.698482+0000","last_became_peered":"2024-05-11T01:01:01.698482+0000","last_unstale":"2024-05-11T01:01:02.793544+0000","last_undegraded":"2024-05-11T01:01:02.793544+0000","last_fullsized":"2024-05-11T01:01:02.793544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:35:54.225824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678878+0000","last_change":"2024-05-11T01:01:00.686573+0000","last_active":"2024-05-11T01:01:02.678878+0000","last_peered":"2024-05-11T01:01:02.678878+0000","last_clean":"2024-05-11T01:01:02.678878+0000","last_became_active":"2024-05-11T01:01:00.686222+0000","last_became_peered":"2024-05-11T01:01:00.686222+0000","last_unstale":"2024-05-11T01:01:02.678878+0000","last_undegraded":"2024-05-11T01:01:02.678878+0000","last_fullsized":"2024-05-11T01:01:02.678878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:20:07.656810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794105+0000","last_change":"2024-05-11T01:01:01.697983+0000","last_active":"2024-05-11T01:01:02.794105+0000","last_peered":"2024-05-11T01:01:02.794105+0000","last_clean":"2024-05-11T01:01:02.794105+0000","last_became_active":"2024-05-11T01:01:01.697645+0000","last_became_peered":"2024-05-11T01:01:01.697645+0000","last_unstale":"2024-05-11T01:01:02.794105+0000","last_undegraded":"2024-05-11T01:01:02.794105+0000","last_fullsized":"2024-05-11T01:01:02.794105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:12.996809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913020+0000","last_change":"2024-05-11T01:01:00.684168+0000","last_active":"2024-05-11T01:01:02.913020+0000","last_peered":"2024-05-11T01:01:02.913020+0000","last_clean":"2024-05-11T01:01:02.913020+0000","last_became_active":"2024-05-11T01:01:00.683839+0000","last_became_peered":"2024-05-11T01:01:00.683839+0000","last_unstale":"2024-05-11T01:01:02.913020+0000","last_undegraded":"2024-05-11T01:01:02.913020+0000","last_fullsized":"2024-05-11T01:01:02.913020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794232+0000","last_change":"2024-05-11T01:01:01.698302+0000","last_active":"2024-05-11T01:01:02.794232+0000","last_peered":"2024-05-11T01:01:02.794232+0000","last_clean":"2024-05-11T01:01:02.794232+0000","last_became_active":"2024-05-11T01:01:01.697989+0000","last_became_peered":"2024-05-11T01:01:01.697989+0000","last_unstale":"2024-05-11T01:01:02.794232+0000","last_undegraded":"2024-05-11T01:01:02.794232+0000","last_fullsized":"2024-05-11T01:01:02.794232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:20:37.893643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703231+0000","last_change":"2024-05-11T01:01:00.681331+0000","last_active":"2024-05-11T01:01:02.703231+0000","last_peered":"2024-05-11T01:01:02.703231+0000","last_clean":"2024-05-11T01:01:02.703231+0000","last_became_active":"2024-05-11T01:01:00.680679+0000","last_became_peered":"2024-05-11T01:01:00.680679+0000","last_unstale":"2024-05-11T01:01:02.703231+0000","last_undegraded":"2024-05-11T01:01:02.703231+0000","last_fullsized":"2024-05-11T01:01:02.703231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678940+0000","last_change":"2024-05-11T01:01:01.687504+0000","last_active":"2024-05-11T01:01:02.678940+0000","last_peered":"2024-05-11T01:01:02.678940+0000","last_clean":"2024-05-11T01:01:02.678940+0000","last_became_active":"2024-05-11T01:01:01.686851+0000","last_became_peered":"2024-05-11T01:01:01.686851+0000","last_unstale":"2024-05-11T01:01:02.678940+0000","last_undegraded":"2024-05-11T01:01:02.678940+0000","last_fullsized":"2024-05-11T01:01:02.678940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:00:03.997273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913094+0000","last_change":"2024-05-11T01:01:00.682797+0000","last_active":"2024-05-11T01:01:02.913094+0000","last_peered":"2024-05-11T01:01:02.913094+0000","last_clean":"2024-05-11T01:01:02.913094+0000","last_became_active":"2024-05-11T01:01:00.682282+0000","last_became_peered":"2024-05-11T01:01:00.682282+0000","last_unstale":"2024-05-11T01:01:02.913094+0000","last_undegraded":"2024-05-11T01:01:02.913094+0000","last_fullsized":"2024-05-11T01:01:02.913094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:00:17.080065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793948+0000","last_change":"2024-05-11T01:01:01.705606+0000","last_active":"2024-05-11T01:01:02.793948+0000","last_peered":"2024-05-11T01:01:02.793948+0000","last_clean":"2024-05-11T01:01:02.793948+0000","last_became_active":"2024-05-11T01:01:01.705324+0000","last_became_peered":"2024-05-11T01:01:01.705324+0000","last_unstale":"2024-05-11T01:01:02.793948+0000","last_undegraded":"2024-05-11T01:01:02.793948+0000","last_fullsized":"2024-05-11T01:01:02.793948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:31.889721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793424+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:01:02.793424+0000","last_peered":"2024-05-11T01:01:02.793424+0000","last_clean":"2024-05-11T01:01:02.793424+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:01:02.793424+0000","last_undegraded":"2024-05-11T01:01:02.793424+0000","last_fullsized":"2024-05-11T01:01:02.793424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677503+0000","last_change":"2024-05-11T01:01:00.679561+0000","last_active":"2024-05-11T01:01:02.677503+0000","last_peered":"2024-05-11T01:01:02.677503+0000","last_clean":"2024-05-11T01:01:02.677503+0000","last_became_active":"2024-05-11T01:01:00.679241+0000","last_became_peered":"2024-05-11T01:01:00.679241+0000","last_unstale":"2024-05-11T01:01:02.677503+0000","last_undegraded":"2024-05-11T01:01:02.677503+0000","last_fullsized":"2024-05-11T01:01:02.677503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:44:24.553921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913407+0000","last_change":"2024-05-11T01:01:01.693541+0000","last_active":"2024-05-11T01:01:02.913407+0000","last_peered":"2024-05-11T01:01:02.913407+0000","last_clean":"2024-05-11T01:01:02.913407+0000","last_became_active":"2024-05-11T01:01:01.693033+0000","last_became_peered":"2024-05-11T01:01:01.693033+0000","last_unstale":"2024-05-11T01:01:02.913407+0000","last_undegraded":"2024-05-11T01:01:02.913407+0000","last_fullsized":"2024-05-11T01:01:02.913407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:14:34.650071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793351+0000","last_change":"2024-05-11T01:01:00.689701+0000","last_active":"2024-05-11T01:01:02.793351+0000","last_peered":"2024-05-11T01:01:02.793351+0000","last_clean":"2024-05-11T01:01:02.793351+0000","last_became_active":"2024-05-11T01:01:00.689439+0000","last_became_peered":"2024-05-11T01:01:00.689439+0000","last_unstale":"2024-05-11T01:01:02.793351+0000","last_undegraded":"2024-05-11T01:01:02.793351+0000","last_fullsized":"2024-05-11T01:01:02.793351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:10:33.194315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.676839+0000","last_change":"2024-05-11T01:01:01.696524+0000","last_active":"2024-05-11T01:01:02.676839+0000","last_peered":"2024-05-11T01:01:02.676839+0000","last_clean":"2024-05-11T01:01:02.676839+0000","last_became_active":"2024-05-11T01:01:01.696245+0000","last_became_peered":"2024-05-11T01:01:01.696245+0000","last_unstale":"2024-05-11T01:01:02.676839+0000","last_undegraded":"2024-05-11T01:01:02.676839+0000","last_fullsized":"2024-05-11T01:01:02.676839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:33.546403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.912909+0000","last_change":"2024-05-11T01:01:00.683932+0000","last_active":"2024-05-11T01:01:02.912909+0000","last_peered":"2024-05-11T01:01:02.912909+0000","last_clean":"2024-05-11T01:01:02.912909+0000","last_became_active":"2024-05-11T01:01:00.683601+0000","last_became_peered":"2024-05-11T01:01:00.683601+0000","last_unstale":"2024-05-11T01:01:02.912909+0000","last_undegraded":"2024-05-11T01:01:02.912909+0000","last_fullsized":"2024-05-11T01:01:02.912909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:59:05.973067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.702979+0000","last_change":"2024-05-11T01:01:01.691231+0000","last_active":"2024-05-11T01:01:02.702979+0000","last_peered":"2024-05-11T01:01:02.702979+0000","last_clean":"2024-05-11T01:01:02.702979+0000","last_became_active":"2024-05-11T01:01:01.690969+0000","last_became_peered":"2024-05-11T01:01:01.690969+0000","last_unstale":"2024-05-11T01:01:02.702979+0000","last_undegraded":"2024-05-11T01:01:02.702979+0000","last_fullsized":"2024-05-11T01:01:02.702979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678402+0000","last_change":"2024-05-11T01:01:00.682079+0000","last_active":"2024-05-11T01:01:02.678402+0000","last_peered":"2024-05-11T01:01:02.678402+0000","last_clean":"2024-05-11T01:01:02.678402+0000","last_became_active":"2024-05-11T01:01:00.681773+0000","last_became_peered":"2024-05-11T01:01:00.681773+0000","last_unstale":"2024-05-11T01:01:02.678402+0000","last_undegraded":"2024-05-11T01:01:02.678402+0000","last_fullsized":"2024-05-11T01:01:02.678402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.679107+0000","last_change":"2024-05-11T01:01:01.695539+0000","last_active":"2024-05-11T01:01:02.679107+0000","last_peered":"2024-05-11T01:01:02.679107+0000","last_clean":"2024-05-11T01:01:02.679107+0000","last_became_active":"2024-05-11T01:01:01.695235+0000","last_became_peered":"2024-05-11T01:01:01.695235+0000","last_unstale":"2024-05-11T01:01:02.679107+0000","last_undegraded":"2024-05-11T01:01:02.679107+0000","last_fullsized":"2024-05-11T01:01:02.679107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:44.383036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703144+0000","last_change":"2024-05-11T01:01:00.680453+0000","last_active":"2024-05-11T01:01:02.703144+0000","last_peered":"2024-05-11T01:01:02.703144+0000","last_clean":"2024-05-11T01:01:02.703144+0000","last_became_active":"2024-05-11T01:01:00.679559+0000","last_became_peered":"2024-05-11T01:01:00.679559+0000","last_unstale":"2024-05-11T01:01:02.703144+0000","last_undegraded":"2024-05-11T01:01:02.703144+0000","last_fullsized":"2024-05-11T01:01:02.703144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:10:33.194315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678735+0000","last_change":"2024-05-11T01:01:01.687198+0000","last_active":"2024-05-11T01:01:02.678735+0000","last_peered":"2024-05-11T01:01:02.678735+0000","last_clean":"2024-05-11T01:01:02.678735+0000","last_became_active":"2024-05-11T01:01:01.686860+0000","last_became_peered":"2024-05-11T01:01:01.686860+0000","last_unstale":"2024-05-11T01:01:02.678735+0000","last_undegraded":"2024-05-11T01:01:02.678735+0000","last_fullsized":"2024-05-11T01:01:02.678735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:31.889721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.794088+0000","last_change":"2024-05-11T01:01:00.688488+0000","last_active":"2024-05-11T01:01:02.794088+0000","last_peered":"2024-05-11T01:01:02.794088+0000","last_clean":"2024-05-11T01:01:02.794088+0000","last_became_active":"2024-05-11T01:01:00.688109+0000","last_became_peered":"2024-05-11T01:01:00.688109+0000","last_unstale":"2024-05-11T01:01:02.794088+0000","last_undegraded":"2024-05-11T01:01:02.794088+0000","last_fullsized":"2024-05-11T01:01:02.794088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:32.542322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703144+0000","last_change":"2024-05-11T01:01:01.689340+0000","last_active":"2024-05-11T01:01:02.703144+0000","last_peered":"2024-05-11T01:01:02.703144+0000","last_clean":"2024-05-11T01:01:02.703144+0000","last_became_active":"2024-05-11T01:01:01.689052+0000","last_became_peered":"2024-05-11T01:01:01.689052+0000","last_unstale":"2024-05-11T01:01:02.703144+0000","last_undegraded":"2024-05-11T01:01:02.703144+0000","last_fullsized":"2024-05-11T01:01:02.703144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703207+0000","last_change":"2024-05-11T01:01:00.678958+0000","last_active":"2024-05-11T01:01:02.703207+0000","last_peered":"2024-05-11T01:01:02.703207+0000","last_clean":"2024-05-11T01:01:02.703207+0000","last_became_active":"2024-05-11T01:01:00.678608+0000","last_became_peered":"2024-05-11T01:01:00.678608+0000","last_unstale":"2024-05-11T01:01:02.703207+0000","last_undegraded":"2024-05-11T01:01:02.703207+0000","last_fullsized":"2024-05-11T01:01:02.703207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:58.764462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913742+0000","last_change":"2024-05-11T01:01:01.686046+0000","last_active":"2024-05-11T01:01:02.913742+0000","last_peered":"2024-05-11T01:01:02.913742+0000","last_clean":"2024-05-11T01:01:02.913742+0000","last_became_active":"2024-05-11T01:01:01.685771+0000","last_became_peered":"2024-05-11T01:01:01.685771+0000","last_unstale":"2024-05-11T01:01:02.913742+0000","last_undegraded":"2024-05-11T01:01:02.913742+0000","last_fullsized":"2024-05-11T01:01:02.913742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:36:33.062657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705667+0000","last_change":"2024-05-11T01:01:00.686048+0000","last_active":"2024-05-11T01:01:02.705667+0000","last_peered":"2024-05-11T01:01:02.705667+0000","last_clean":"2024-05-11T01:01:02.705667+0000","last_became_active":"2024-05-11T01:01:00.685215+0000","last_became_peered":"2024-05-11T01:01:00.685215+0000","last_unstale":"2024-05-11T01:01:02.705667+0000","last_undegraded":"2024-05-11T01:01:02.705667+0000","last_fullsized":"2024-05-11T01:01:02.705667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677114+0000","last_change":"2024-05-11T01:01:01.695783+0000","last_active":"2024-05-11T01:01:02.677114+0000","last_peered":"2024-05-11T01:01:02.677114+0000","last_clean":"2024-05-11T01:01:02.677114+0000","last_became_active":"2024-05-11T01:01:01.695264+0000","last_became_peered":"2024-05-11T01:01:01.695264+0000","last_unstale":"2024-05-11T01:01:02.677114+0000","last_undegraded":"2024-05-11T01:01:02.677114+0000","last_fullsized":"2024-05-11T01:01:02.677114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:33:27.544203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793321+0000","last_change":"2024-05-11T01:01:00.698030+0000","last_active":"2024-05-11T01:01:02.793321+0000","last_peered":"2024-05-11T01:01:02.793321+0000","last_clean":"2024-05-11T01:01:02.793321+0000","last_became_active":"2024-05-11T01:01:00.697776+0000","last_became_peered":"2024-05-11T01:01:00.697776+0000","last_unstale":"2024-05-11T01:01:02.793321+0000","last_undegraded":"2024-05-11T01:01:02.793321+0000","last_fullsized":"2024-05-11T01:01:02.793321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677953+0000","last_change":"2024-05-11T01:01:01.696098+0000","last_active":"2024-05-11T01:01:02.677953+0000","last_peered":"2024-05-11T01:01:02.677953+0000","last_clean":"2024-05-11T01:01:02.677953+0000","last_became_active":"2024-05-11T01:01:01.695670+0000","last_became_peered":"2024-05-11T01:01:01.695670+0000","last_unstale":"2024-05-11T01:01:02.677953+0000","last_undegraded":"2024-05-11T01:01:02.677953+0000","last_fullsized":"2024-05-11T01:01:02.677953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:49.477561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703023+0000","last_change":"2024-05-11T01:01:00.680814+0000","last_active":"2024-05-11T01:01:02.703023+0000","last_peered":"2024-05-11T01:01:02.703023+0000","last_clean":"2024-05-11T01:01:02.703023+0000","last_became_active":"2024-05-11T01:01:00.680415+0000","last_became_peered":"2024-05-11T01:01:00.680415+0000","last_unstale":"2024-05-11T01:01:02.703023+0000","last_undegraded":"2024-05-11T01:01:02.703023+0000","last_fullsized":"2024-05-11T01:01:02.703023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:32:43.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913100+0000","last_change":"2024-05-11T01:01:01.690240+0000","last_active":"2024-05-11T01:01:02.913100+0000","last_peered":"2024-05-11T01:01:02.913100+0000","last_clean":"2024-05-11T01:01:02.913100+0000","last_became_active":"2024-05-11T01:01:01.689938+0000","last_became_peered":"2024-05-11T01:01:01.689938+0000","last_unstale":"2024-05-11T01:01:02.913100+0000","last_undegraded":"2024-05-11T01:01:02.913100+0000","last_fullsized":"2024-05-11T01:01:02.913100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:10:39.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.703036+0000","last_change":"2024-05-11T01:01:00.682185+0000","last_active":"2024-05-11T01:01:02.703036+0000","last_peered":"2024-05-11T01:01:02.703036+0000","last_clean":"2024-05-11T01:01:02.703036+0000","last_became_active":"2024-05-11T01:01:00.681773+0000","last_became_peered":"2024-05-11T01:01:00.681773+0000","last_unstale":"2024-05-11T01:01:02.703036+0000","last_undegraded":"2024-05-11T01:01:02.703036+0000","last_fullsized":"2024-05-11T01:01:02.703036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:45:48.473480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913467+0000","last_change":"2024-05-11T01:01:01.688107+0000","last_active":"2024-05-11T01:01:02.913467+0000","last_peered":"2024-05-11T01:01:02.913467+0000","last_clean":"2024-05-11T01:01:02.913467+0000","last_became_active":"2024-05-11T01:01:01.686201+0000","last_became_peered":"2024-05-11T01:01:01.686201+0000","last_unstale":"2024-05-11T01:01:02.913467+0000","last_undegraded":"2024-05-11T01:01:02.913467+0000","last_fullsized":"2024-05-11T01:01:02.913467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:21:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793707+0000","last_change":"2024-05-11T01:01:00.683637+0000","last_active":"2024-05-11T01:01:02.793707+0000","last_peered":"2024-05-11T01:01:02.793707+0000","last_clean":"2024-05-11T01:01:02.793707+0000","last_became_active":"2024-05-11T01:01:00.683373+0000","last_became_peered":"2024-05-11T01:01:00.683373+0000","last_unstale":"2024-05-11T01:01:02.793707+0000","last_undegraded":"2024-05-11T01:01:02.793707+0000","last_fullsized":"2024-05-11T01:01:02.793707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:38:58.764462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.705465+0000","last_change":"2024-05-11T01:01:01.695407+0000","last_active":"2024-05-11T01:01:02.705465+0000","last_peered":"2024-05-11T01:01:02.705465+0000","last_clean":"2024-05-11T01:01:02.705465+0000","last_became_active":"2024-05-11T01:01:01.695012+0000","last_became_peered":"2024-05-11T01:01:01.695012+0000","last_unstale":"2024-05-11T01:01:02.705465+0000","last_undegraded":"2024-05-11T01:01:02.705465+0000","last_fullsized":"2024-05-11T01:01:02.705465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678475+0000","last_change":"2024-05-11T01:01:01.698832+0000","last_active":"2024-05-11T01:01:02.678475+0000","last_peered":"2024-05-11T01:01:02.678475+0000","last_clean":"2024-05-11T01:01:02.678475+0000","last_became_active":"2024-05-11T01:01:01.698487+0000","last_became_peered":"2024-05-11T01:01:01.698487+0000","last_unstale":"2024-05-11T01:01:02.678475+0000","last_undegraded":"2024-05-11T01:01:02.678475+0000","last_fullsized":"2024-05-11T01:01:02.678475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:17.887457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793796+0000","last_change":"2024-05-11T01:01:00.690097+0000","last_active":"2024-05-11T01:01:02.793796+0000","last_peered":"2024-05-11T01:01:02.793796+0000","last_clean":"2024-05-11T01:01:02.793796+0000","last_became_active":"2024-05-11T01:01:00.689772+0000","last_became_peered":"2024-05-11T01:01:00.689772+0000","last_unstale":"2024-05-11T01:01:02.793796+0000","last_undegraded":"2024-05-11T01:01:02.793796+0000","last_fullsized":"2024-05-11T01:01:02.793796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:16.696278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.678776+0000","last_change":"2024-05-11T01:01:01.699646+0000","last_active":"2024-05-11T01:01:02.678776+0000","last_peered":"2024-05-11T01:01:02.678776+0000","last_clean":"2024-05-11T01:01:02.678776+0000","last_became_active":"2024-05-11T01:01:01.699388+0000","last_became_peered":"2024-05-11T01:01:01.699388+0000","last_unstale":"2024-05-11T01:01:02.678776+0000","last_undegraded":"2024-05-11T01:01:02.678776+0000","last_fullsized":"2024-05-11T01:01:02.678776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:12.996809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.793982+0000","last_change":"2024-05-11T01:01:00.688793+0000","last_active":"2024-05-11T01:01:02.793982+0000","last_peered":"2024-05-11T01:01:02.793982+0000","last_clean":"2024-05-11T01:01:02.793982+0000","last_became_active":"2024-05-11T01:01:00.688279+0000","last_became_peered":"2024-05-11T01:01:00.688279+0000","last_unstale":"2024-05-11T01:01:02.793982+0000","last_undegraded":"2024-05-11T01:01:02.793982+0000","last_fullsized":"2024-05-11T01:01:02.793982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:00:50.870500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677179+0000","last_change":"2024-05-11T01:01:01.682113+0000","last_active":"2024-05-11T01:01:02.677179+0000","last_peered":"2024-05-11T01:01:02.677179+0000","last_clean":"2024-05-11T01:01:02.677179+0000","last_became_active":"2024-05-11T01:01:01.681706+0000","last_became_peered":"2024-05-11T01:01:01.681706+0000","last_unstale":"2024-05-11T01:01:02.677179+0000","last_undegraded":"2024-05-11T01:01:02.677179+0000","last_fullsized":"2024-05-11T01:01:02.677179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:52:25.934935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913646+0000","last_change":"2024-05-11T01:01:00.680118+0000","last_active":"2024-05-11T01:01:02.913646+0000","last_peered":"2024-05-11T01:01:02.913646+0000","last_clean":"2024-05-11T01:01:02.913646+0000","last_became_active":"2024-05-11T01:01:00.679809+0000","last_became_peered":"2024-05-11T01:01:00.679809+0000","last_unstale":"2024-05-11T01:01:02.913646+0000","last_undegraded":"2024-05-11T01:01:02.913646+0000","last_fullsized":"2024-05-11T01:01:02.913646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:12:31.764619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.913693+0000","last_change":"2024-05-11T01:01:01.692697+0000","last_active":"2024-05-11T01:01:02.913693+0000","last_peered":"2024-05-11T01:01:02.913693+0000","last_clean":"2024-05-11T01:01:02.913693+0000","last_became_active":"2024-05-11T01:01:01.691026+0000","last_became_peered":"2024-05-11T01:01:01.691026+0000","last_unstale":"2024-05-11T01:01:02.913693+0000","last_undegraded":"2024-05-11T01:01:02.913693+0000","last_fullsized":"2024-05-11T01:01:02.913693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:00.661168+0000","last_clean_scrub_stamp":"2024-05-11T01:01:00.661168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:18:31.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-11T01:01:02.677208+0000","last_change":"2024-05-11T01:01:00.690191+0000","last_active":"2024-05-11T01:01:02.677208+0000","last_peered":"2024-05-11T01:01:02.677208+0000","last_clean":"2024-05-11T01:01:02.677208+0000","last_became_active":"2024-05-11T01:01:00.689804+0000","last_became_peered":"2024-05-11T01:01:00.689804+0000","last_unstale":"2024-05-11T01:01:02.677208+0000","last_undegraded":"2024-05-11T01:01:02.677208+0000","last_fullsized":"2024-05-11T01:01:02.677208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:59.657087+0000","last_clean_scrub_stamp":"2024-05-11T01:00:59.657087+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:13.659371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:01:15.165 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T01:01:15.165 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-11T01:01:15.165 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-11T01:01:15.165 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-11T01:01:15.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.306+0000 7fbc9bfe4640 1 Processor -- start 2024-05-11T01:01:15.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.306+0000 7fbc9bfe4640 1 -- start start 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc9bfe4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc9415f530 0x7fbc940cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc9bfe4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc940d3eb0 0x7fbc940d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc9bfe4640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc940d0510 con 0x7fbc940d15b0 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc9bfe4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fbc940d5d00 con 0x7fbc9415f530 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc9bfe4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbc940d07f0 con 0x7fbc940d3eb0 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc99d80640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc9415f530 0x7fbc940cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc9957f640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbc940d15b0 0x7fbc940d37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58168/0 (socket says 172.21.15.52:58168) 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc99d80640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbc9415f530 0x7fbc940cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35408/0 (socket says 172.21.15.52:35408) 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc9957f640 1 -- 172.21.15.52:0/2053522198 learned_addr learned my addr 172.21.15.52:0/2053522198 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.307+0000 7fbc9a581640 1 --2- 172.21.15.52:0/2053522198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc940d3eb0 0x7fbc940d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.308+0000 7fbc99d80640 1 -- 172.21.15.52:0/2053522198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc940d3eb0 msgr2=0x7fbc940d57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.308+0000 7fbc99d80640 1 --2- 172.21.15.52:0/2053522198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc940d3eb0 0x7fbc940d57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.308+0000 7fbc99d80640 1 -- 172.21.15.52:0/2053522198 >> v1:172.21.15.52:6789/0 conn(0x7fbc940d15b0 legacy=0x7fbc940d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.308+0000 7fbc99d80640 1 -- 172.21.15.52:0/2053522198 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc940d0ad0 con 0x7fbc9415f530 2024-05-11T01:01:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.308+0000 7fbc98d7e640 1 -- 172.21.15.52:0/2053522198 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370601298 0 0) 0x7fbc940d0510 con 0x7fbc940d15b0 2024-05-11T01:01:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.308+0000 7fbc9a581640 1 --2- 172.21.15.52:0/2053522198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc940d3eb0 0x7fbc940d57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.308+0000 7fbc99d80640 1 --2- 172.21.15.52:0/2053522198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc9415f530 0x7fbc940cfe30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc900097a0 tx=0x7fbc9002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c43ce8a5e4bf8c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.308+0000 7fbc98d7e640 1 -- 172.21.15.52:0/2053522198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90037070 con 0x7fbc9415f530 2024-05-11T01:01:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc98d7e640 1 -- 172.21.15.52:0/2053522198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc90032440 con 0x7fbc9415f530 2024-05-11T01:01:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc98d7e640 1 -- 172.21.15.52:0/2053522198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90032810 con 0x7fbc9415f530 2024-05-11T01:01:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/2053522198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc9415f530 msgr2=0x7fbc940cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc9bfe4640 1 --2- 172.21.15.52:0/2053522198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc9415f530 0x7fbc940cfe30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc900097a0 tx=0x7fbc9002a180 comp rx=0 tx=0).stop 2024-05-11T01:01:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/2053522198 shutdown_connections 2024-05-11T01:01:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc9bfe4640 1 --2- 172.21.15.52:0/2053522198 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc940d3eb0 0x7fbc940d57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc9bfe4640 1 --2- 172.21.15.52:0/2053522198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc9415f530 0x7fbc940cfe30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/2053522198 >> 172.21.15.52:0/2053522198 conn(0x7fbc9408cfb0 msgr2=0x7fbc9409e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/2053522198 shutdown_connections 2024-05-11T01:01:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.309+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/2053522198 wait complete. 2024-05-11T01:01:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.310+0000 7fbc9bfe4640 1 Processor -- start 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.310+0000 7fbc9bfe4640 1 -- start start 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc9bfe4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc940d15b0 0x7fbc940ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc9bfe4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc940d3eb0 0x7fbc94153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc9bfe4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc9415f530 0x7fbc9415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc9bfe4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9415f910 con 0x7fbc940d15b0 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc9bfe4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbc940d1090 con 0x7fbc940d3eb0 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc9bfe4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbc9415fa90 con 0x7fbc9415f530 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc99d80640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc940d15b0 0x7fbc940ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc99d80640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc940d15b0 0x7fbc940ba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35424/0 (socket says 172.21.15.52:35424) 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc99d80640 1 -- 172.21.15.52:0/1643266433 learned_addr learned my addr 172.21.15.52:0/1643266433 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc99d80640 1 -- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc9415f530 msgr2=0x7fbc9415fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc99d80640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc9415f530 0x7fbc9415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc99d80640 1 -- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc940d3eb0 msgr2=0x7fbc94153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc99d80640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc940d3eb0 0x7fbc94153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.311+0000 7fbc99d80640 1 -- 172.21.15.52:0/1643266433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc94153be0 con 0x7fbc940d15b0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.312+0000 7fbc99d80640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc940d15b0 0x7fbc940ba6a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbc90000c00 tx=0x7fbc900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.312+0000 7fbc8a7fc640 1 -- 172.21.15.52:0/1643266433 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90037040 con 0x7fbc940d15b0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.312+0000 7fbc8a7fc640 1 -- 172.21.15.52:0/1643266433 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc90042070 con 0x7fbc940d15b0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.312+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc94153ec0 con 0x7fbc940d15b0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.312+0000 7fbc8a7fc640 1 -- 172.21.15.52:0/1643266433 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90030bb0 con 0x7fbc940d15b0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.312+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc94154480 con 0x7fbc940d15b0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.313+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc940696a0 con 0x7fbc940d15b0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.313+0000 7fbc8a7fc640 1 -- 172.21.15.52:0/1643266433 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbc90047460 con 0x7fbc940d15b0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.314+0000 7fbc8a7fc640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc780762d0 0x7fbc78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.314+0000 7fbc8a7fc640 1 -- 172.21.15.52:0/1643266433 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc900b8130 con 0x7fbc940d15b0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.314+0000 7fbc9957f640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc780762d0 0x7fbc78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.315+0000 7fbc9957f640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc780762d0 0x7fbc78078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbc84000c00 tx=0x7fbc840078d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:15.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.318+0000 7fbc8a7fc640 1 -- 172.21.15.52:0/1643266433 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbc90032e20 con 0x7fbc940d15b0 2024-05-11T01:01:15.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.547+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbc940d1d50 con 0x7fbc940d15b0 2024-05-11T01:01:15.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.547+0000 7fbc8a7fc640 1 -- 172.21.15.52:0/1643266433 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbc90081b20 con 0x7fbc940d15b0 2024-05-11T01:01:15.549 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:15.549 INFO:teuthology.orchestra.run.smithi052.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc780762d0 msgr2=0x7fbc78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc780762d0 0x7fbc78078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbc84000c00 tx=0x7fbc840078d0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc940d15b0 msgr2=0x7fbc940ba6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc940d15b0 0x7fbc940ba6a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbc90000c00 tx=0x7fbc900046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 shutdown_connections 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbc780762d0 0x7fbc78078790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc9415f530 0x7fbc9415fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc940d3eb0 0x7fbc94153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 --2- 172.21.15.52:0/1643266433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc940d15b0 0x7fbc940ba6a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 >> 172.21.15.52:0/1643266433 conn(0x7fbc9408cfb0 msgr2=0x7fbc940d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 shutdown_connections 2024-05-11T01:01:15.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.549+0000 7fbc9bfe4640 1 -- 172.21.15.52:0/1643266433 wait complete. 2024-05-11T01:01:15.561 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-11T01:01:15.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:15.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.665+0000 7fe768c3f640 1 Processor -- start 2024-05-11T01:01:15.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.665+0000 7fe768c3f640 1 -- start start 2024-05-11T01:01:15.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe768c3f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe76415f430 0x7fe76415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe768c3f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe7640d3d90 0x7fe7640d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe768c3f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe764163250 con 0x7fe7641602b0 2024-05-11T01:01:15.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe768c3f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe764163530 con 0x7fe7640d3d90 2024-05-11T01:01:15.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe768c3f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe764163810 con 0x7fe76415f430 2024-05-11T01:01:15.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe762d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe7640d3d90 0x7fe7640d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe759d9b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fe7641602b0 0x7fe7640d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58174/0 (socket says 172.21.15.52:58174) 2024-05-11T01:01:15.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe759d9b640 1 -- 172.21.15.52:0/3123785465 learned_addr learned my addr 172.21.15.52:0/3123785465 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:15.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe762d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe7640d3d90 0x7fe7640d2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35426/0 (socket says 172.21.15.52:35426) 2024-05-11T01:01:15.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe76259c640 1 --2- 172.21.15.52:0/3123785465 >> v2:172.21.15.119:3300/0 conn(0x7fe76415f430 0x7fe76415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe762d9d640 1 -- 172.21.15.52:0/3123785465 >> v2:172.21.15.119:3300/0 conn(0x7fe76415f430 msgr2=0x7fe76415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:15.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe762d9d640 1 --2- 172.21.15.52:0/3123785465 >> v2:172.21.15.119:3300/0 conn(0x7fe76415f430 0x7fe76415fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe762d9d640 1 -- 172.21.15.52:0/3123785465 >> v1:172.21.15.52:6789/0 conn(0x7fe7641602b0 legacy=0x7fe7640d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe762d9d640 1 -- 172.21.15.52:0/3123785465 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe764163af0 con 0x7fe7640d3d90 2024-05-11T01:01:15.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.666+0000 7fe761d9b640 1 -- 172.21.15.52:0/3123785465 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 604101102 0 0) 0x7fe764163250 con 0x7fe7641602b0 2024-05-11T01:01:15.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.667+0000 7fe762d9d640 1 --2- 172.21.15.52:0/3123785465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7640d3d90 0x7fe7640d2260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7640b6b80 tx=0x7fe75c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536e1cb9f688d7f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:15.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.667+0000 7fe761d9b640 1 -- 172.21.15.52:0/3123785465 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c007740 con 0x7fe7640d3d90 2024-05-11T01:01:15.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.667+0000 7fe761d9b640 1 -- 172.21.15.52:0/3123785465 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c0078e0 con 0x7fe7640d3d90 2024-05-11T01:01:15.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.667+0000 7fe761d9b640 1 -- 172.21.15.52:0/3123785465 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c007cb0 con 0x7fe7640d3d90 2024-05-11T01:01:15.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.667+0000 7fe768c3f640 1 -- 172.21.15.52:0/3123785465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7640d3d90 msgr2=0x7fe7640d2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.667+0000 7fe768c3f640 1 --2- 172.21.15.52:0/3123785465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7640d3d90 0x7fe7640d2260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7640b6b80 tx=0x7fe75c02a680 comp rx=0 tx=0).stop 2024-05-11T01:01:15.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.668+0000 7fe768c3f640 1 -- 172.21.15.52:0/3123785465 shutdown_connections 2024-05-11T01:01:15.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.668+0000 7fe768c3f640 1 --2- 172.21.15.52:0/3123785465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7640d3d90 0x7fe7640d2260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.668+0000 7fe768c3f640 1 --2- 172.21.15.52:0/3123785465 >> v2:172.21.15.119:3300/0 conn(0x7fe76415f430 0x7fe76415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.668+0000 7fe768c3f640 1 -- 172.21.15.52:0/3123785465 >> 172.21.15.52:0/3123785465 conn(0x7fe76408eb70 msgr2=0x7fe76409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:15.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.668+0000 7fe768c3f640 1 -- 172.21.15.52:0/3123785465 shutdown_connections 2024-05-11T01:01:15.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.668+0000 7fe768c3f640 1 -- 172.21.15.52:0/3123785465 wait complete. 2024-05-11T01:01:15.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.669+0000 7fe768c3f640 1 Processor -- start 2024-05-11T01:01:15.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.669+0000 7fe768c3f640 1 -- start start 2024-05-11T01:01:15.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe768c3f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640d3d90 0x7fe7640ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe768c3f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe76415f430 0x7fe76415d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe768c3f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7641602b0 0x7fe7640ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe768c3f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe764163030 con 0x7fe7641602b0 2024-05-11T01:01:15.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe768c3f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe764162eb0 con 0x7fe7640d3d90 2024-05-11T01:01:15.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe768c3f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe7640d27a0 con 0x7fe76415f430 2024-05-11T01:01:15.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe762d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7641602b0 0x7fe7640ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe762d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7641602b0 0x7fe7640ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35428/0 (socket says 172.21.15.52:35428) 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe762d9d640 1 -- 172.21.15.52:0/2423791895 learned_addr learned my addr 172.21.15.52:0/2423791895 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe759d9b640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe76415f430 0x7fe76415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe76259c640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640d3d90 0x7fe7640ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe762d9d640 1 -- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe76415f430 msgr2=0x7fe76415d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe762d9d640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe76415f430 0x7fe76415d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe762d9d640 1 -- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640d3d90 msgr2=0x7fe7640ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe762d9d640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640d3d90 0x7fe7640ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe762d9d640 1 -- 172.21.15.52:0/2423791895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640cd470 con 0x7fe7641602b0 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.670+0000 7fe76259c640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640d3d90 0x7fe7640ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.671+0000 7fe762d9d640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7641602b0 0x7fe7640ca250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe75c02ac70 tx=0x7fe75c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.671+0000 7fe75b7fe640 1 -- 172.21.15.52:0/2423791895 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c040070 con 0x7fe7641602b0 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.671+0000 7fe75b7fe640 1 -- 172.21.15.52:0/2423791895 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c031bc0 con 0x7fe7641602b0 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.671+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7640cd750 con 0x7fe7641602b0 2024-05-11T01:01:15.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.671+0000 7fe75b7fe640 1 -- 172.21.15.52:0/2423791895 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c045460 con 0x7fe7641602b0 2024-05-11T01:01:15.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.671+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7640cd160 con 0x7fe7641602b0 2024-05-11T01:01:15.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.672+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7640cdc70 con 0x7fe7641602b0 2024-05-11T01:01:15.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.672+0000 7fe75b7fe640 1 -- 172.21.15.52:0/2423791895 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe75c039040 con 0x7fe7641602b0 2024-05-11T01:01:15.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.673+0000 7fe75b7fe640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe744076300 0x7fe7440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:15.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.673+0000 7fe75b7fe640 1 -- 172.21.15.52:0/2423791895 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe75c0b8f00 con 0x7fe7641602b0 2024-05-11T01:01:15.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.673+0000 7fe76259c640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe744076300 0x7fe7440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:15.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.673+0000 7fe76259c640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe744076300 0x7fe7440787c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe754006fd0 tx=0x7fe754008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:15.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.677+0000 7fe75b7fe640 1 -- 172.21.15.52:0/2423791895 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe7640cdc70 con 0x7fe7641602b0 2024-05-11T01:01:15.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.912+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7641f5240 con 0x7fe7641602b0 2024-05-11T01:01:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.912+0000 7fe75b7fe640 1 -- 172.21.15.52:0/2423791895 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fe75c082850 con 0x7fe7641602b0 2024-05-11T01:01:15.914 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:15.914 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2245862597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2245862597},{"type":"v1","addr":"172.21.15.119:6837","nonce":2245862597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4286578298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4286578298},{"type":"v1","addr":"172.21.15.52:6835","nonce":4286578298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:01.052672+0000","modified":"2024-05-11T01:01:04.065799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/1484114523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1484114523},{"type":"v1","addr":"172.21.15.119:6836","nonce":1484114523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-11T01:01:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.914+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe744076300 msgr2=0x7fe7440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.914+0000 7fe768c3f640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe744076300 0x7fe7440787c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe754006fd0 tx=0x7fe754008040 comp rx=0 tx=0).stop 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.914+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7641602b0 msgr2=0x7fe7640ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.914+0000 7fe768c3f640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7641602b0 0x7fe7640ca250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe75c02ac70 tx=0x7fe75c004a40 comp rx=0 tx=0).stop 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.914+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 shutdown_connections 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.914+0000 7fe768c3f640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe744076300 0x7fe7440787c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.914+0000 7fe768c3f640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7641602b0 0x7fe7640ca250 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.914+0000 7fe768c3f640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe76415f430 0x7fe76415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.915+0000 7fe768c3f640 1 --2- 172.21.15.52:0/2423791895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7640d3d90 0x7fe7640ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.915+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 >> 172.21.15.52:0/2423791895 conn(0x7fe76408eb70 msgr2=0x7fe76409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.915+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 shutdown_connections 2024-05-11T01:01:15.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:15.915+0000 7fe768c3f640 1 -- 172.21.15.52:0/2423791895 wait complete. 2024-05-11T01:01:15.927 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:16.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.031+0000 7f3b169f0640 1 Processor -- start 2024-05-11T01:01:16.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.031+0000 7f3b169f0640 1 -- start start 2024-05-11T01:01:16.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.032+0000 7f3b169f0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b1009c130 0x7f3b100b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.032+0000 7f3b169f0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3b10160e00 0x7f3b100b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.032+0000 7f3b169f0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b100d1000 con 0x7f3b10160a20 2024-05-11T01:01:16.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.032+0000 7f3b169f0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f3b100d20a0 con 0x7f3b10160e00 2024-05-11T01:01:16.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.032+0000 7f3b169f0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3b100d12e0 con 0x7f3b1009c130 2024-05-11T01:01:16.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.032+0000 7f3b14f8d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3b10160e00 0x7f3b100b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.032+0000 7f3b14f8d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3b10160e00 0x7f3b100b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35434/0 (socket says 172.21.15.52:35434) 2024-05-11T01:01:16.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.032+0000 7f3b14f8d640 1 -- 172.21.15.52:0/2997304578 learned_addr learned my addr 172.21.15.52:0/2997304578 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:16.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.033+0000 7f3b0f7fe640 1 --2- 172.21.15.52:0/2997304578 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b1009c130 0x7f3b100b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.033+0000 7f3b14f8d640 1 -- 172.21.15.52:0/2997304578 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b1009c130 msgr2=0x7f3b100b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.033+0000 7f3b14f8d640 1 --2- 172.21.15.52:0/2997304578 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b1009c130 0x7f3b100b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.033+0000 7f3b14f8d640 1 -- 172.21.15.52:0/2997304578 >> v1:172.21.15.119:6789/0 conn(0x7f3b10160a20 legacy=0x7f3b1009ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.033+0000 7f3b14f8d640 1 -- 172.21.15.52:0/2997304578 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b100d15c0 con 0x7f3b10160e00 2024-05-11T01:01:16.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.033+0000 7f3b0f7fe640 1 --2- 172.21.15.52:0/2997304578 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b1009c130 0x7f3b100b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.033+0000 7f3b14f8d640 1 --2- 172.21.15.52:0/2997304578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b10160e00 0x7f3b100b55b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83069902c036d42a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b0effd640 1 -- 172.21.15.52:0/2997304578 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08036070 con 0x7f3b10160e00 2024-05-11T01:01:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b0effd640 1 -- 172.21.15.52:0/2997304578 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b08031600 con 0x7f3b10160e00 2024-05-11T01:01:16.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b0effd640 1 -- 172.21.15.52:0/2997304578 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b080319d0 con 0x7f3b10160e00 2024-05-11T01:01:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b169f0640 1 -- 172.21.15.52:0/2997304578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b10160e00 msgr2=0x7f3b100b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b169f0640 1 --2- 172.21.15.52:0/2997304578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b10160e00 0x7f3b100b55b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0802a5f0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b169f0640 1 -- 172.21.15.52:0/2997304578 shutdown_connections 2024-05-11T01:01:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b169f0640 1 --2- 172.21.15.52:0/2997304578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b10160e00 0x7f3b100b55b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b169f0640 1 --2- 172.21.15.52:0/2997304578 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b1009c130 0x7f3b100b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b169f0640 1 -- 172.21.15.52:0/2997304578 >> 172.21.15.52:0/2997304578 conn(0x7f3b100b4a90 msgr2=0x7f3b100b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b169f0640 1 -- 172.21.15.52:0/2997304578 shutdown_connections 2024-05-11T01:01:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.034+0000 7f3b169f0640 1 -- 172.21.15.52:0/2997304578 wait complete. 2024-05-11T01:01:16.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.035+0000 7f3b169f0640 1 Processor -- start 2024-05-11T01:01:16.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.036+0000 7f3b169f0640 1 -- start start 2024-05-11T01:01:16.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.036+0000 7f3b169f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1009c130 0x7f3b100719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.036+0000 7f3b169f0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b10160a20 0x7f3b100c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.036+0000 7f3b169f0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b10160e00 0x7f3b100cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.036+0000 7f3b169f0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b100cdb70 con 0x7f3b1009c130 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.036+0000 7f3b169f0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3b100d1b80 con 0x7f3b10160a20 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.036+0000 7f3b169f0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3b100cdcf0 con 0x7f3b10160e00 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1009c130 0x7f3b100719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1009c130 0x7f3b100719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35444/0 (socket says 172.21.15.52:35444) 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0ffff640 1 -- 172.21.15.52:0/2238232589 learned_addr learned my addr 172.21.15.52:0/2238232589 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0ffff640 1 -- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b10160e00 msgr2=0x7f3b100cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0f7fe640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b10160a20 0x7f3b100c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0ffff640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b10160e00 0x7f3b100cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0ffff640 1 -- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b10160a20 msgr2=0x7f3b100c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0ffff640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b10160a20 0x7f3b100c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0ffff640 1 -- 172.21.15.52:0/2238232589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1006e140 con 0x7f3b1009c130 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0ffff640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1009c130 0x7f3b100719b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c430 tx=0x7f3b0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0cff9640 1 -- 172.21.15.52:0/2238232589 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00011790 con 0x7f3b1009c130 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0cff9640 1 -- 172.21.15.52:0/2238232589 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b00017070 con 0x7f3b1009c130 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1006e420 con 0x7f3b1009c130 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b0cff9640 1 -- 172.21.15.52:0/2238232589 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00011bb0 con 0x7f3b1009c130 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.037+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1006e9e0 con 0x7f3b1009c130 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.038+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b10069780 con 0x7f3b1009c130 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.039+0000 7f3b0cff9640 1 -- 172.21.15.52:0/2238232589 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b0001c460 con 0x7f3b1009c130 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.039+0000 7f3b0cff9640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3afc0762d0 0x7f3afc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.039+0000 7f3b0cff9640 1 -- 172.21.15.52:0/2238232589 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3b00097460 con 0x7f3b1009c130 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.040+0000 7f3b0f7fe640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3afc0762d0 0x7f3afc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.040+0000 7f3b0f7fe640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3afc0762d0 0x7f3afc078790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b5c30 tx=0x7f3b04008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:16.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.045+0000 7f3b0cff9640 1 -- 172.21.15.52:0/2238232589 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3b00060dc0 con 0x7f3b1009c130 2024-05-11T01:01:16.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.278+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b101600e0 con 0x7f3b1009c130 2024-05-11T01:01:16.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.279+0000 7f3b0cff9640 1 -- 172.21.15.52:0/2238232589 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f3b000606c0 con 0x7f3b1009c130 2024-05-11T01:01:16.280 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:16.280 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2245862597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2245862597},{"type":"v1","addr":"172.21.15.119:6837","nonce":2245862597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4286578298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4286578298},{"type":"v1","addr":"172.21.15.52:6835","nonce":4286578298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:01.052672+0000","modified":"2024-05-11T01:01:04.065799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/1484114523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1484114523},{"type":"v1","addr":"172.21.15.119:6836","nonce":1484114523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-11T01:01:16.280 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-05-11T01:01:16.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.280+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3afc0762d0 msgr2=0x7f3afc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.280+0000 7f3b169f0640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3afc0762d0 0x7f3afc078790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b5c30 tx=0x7f3b04008040 comp rx=0 tx=0).stop 2024-05-11T01:01:16.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1009c130 msgr2=0x7f3b100719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1009c130 0x7f3b100719b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c430 tx=0x7f3b0000c8f0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 shutdown_connections 2024-05-11T01:01:16.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3afc0762d0 0x7f3afc078790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b10160e00 0x7f3b100cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b10160a20 0x7f3b100c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 --2- 172.21.15.52:0/2238232589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1009c130 0x7f3b100719b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 >> 172.21.15.52:0/2238232589 conn(0x7f3b100b4a90 msgr2=0x7f3b100b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:16.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 shutdown_connections 2024-05-11T01:01:16.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.281+0000 7f3b169f0640 1 -- 172.21.15.52:0/2238232589 wait complete. 2024-05-11T01:01:16.293 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:16.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.396+0000 7f8a1c1f7640 1 Processor -- start 2024-05-11T01:01:16.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.396+0000 7f8a1c1f7640 1 -- start start 2024-05-11T01:01:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.396+0000 7f8a1c1f7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8a1415f430 0x7f8a1415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.396+0000 7f8a1c1f7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a141602b0 0x7f8a141606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.396+0000 7f8a1c1f7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a140d1b50 0x7f8a140d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.396+0000 7f8a1c1f7640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8a14163250 con 0x7f8a1415f430 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.396+0000 7f8a1c1f7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8a14163530 con 0x7f8a140d1b50 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.396+0000 7f8a1c1f7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a14163810 con 0x7f8a141602b0 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a19f93640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8a1415f430 0x7f8a1415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a19f93640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8a1415f430 0x7f8a1415fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35448/0 (socket says 172.21.15.52:35448) 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a19f93640 1 -- 172.21.15.52:0/387414024 learned_addr learned my addr 172.21.15.52:0/387414024 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a19792640 1 --2- 172.21.15.52:0/387414024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a141602b0 0x7f8a141606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a1a794640 1 --2- 172.21.15.52:0/387414024 >> v2:172.21.15.119:3300/0 conn(0x7f8a140d1b50 0x7f8a140d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a19f93640 1 -- 172.21.15.52:0/387414024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a141602b0 msgr2=0x7f8a141606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a19f93640 1 --2- 172.21.15.52:0/387414024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a141602b0 0x7f8a141606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a19f93640 1 -- 172.21.15.52:0/387414024 >> v2:172.21.15.119:3300/0 conn(0x7f8a140d1b50 msgr2=0x7f8a140d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:16.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a19f93640 1 --2- 172.21.15.52:0/387414024 >> v2:172.21.15.119:3300/0 conn(0x7f8a140d1b50 0x7f8a140d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.397+0000 7f8a19f93640 1 -- 172.21.15.52:0/387414024 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a14163af0 con 0x7f8a1415f430 2024-05-11T01:01:16.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a19f93640 1 --2- 172.21.15.52:0/387414024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a1415f430 0x7f8a1415fd70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a140cf710 tx=0x7f8a1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9028e6d699f4dceb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a18f91640 1 -- 172.21.15.52:0/387414024 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10037070 con 0x7f8a1415f430 2024-05-11T01:01:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a18f91640 1 -- 172.21.15.52:0/387414024 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a10032440 con 0x7f8a1415f430 2024-05-11T01:01:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a18f91640 1 -- 172.21.15.52:0/387414024 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10032810 con 0x7f8a1415f430 2024-05-11T01:01:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/387414024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a1415f430 msgr2=0x7f8a1415fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/387414024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a1415f430 0x7f8a1415fd70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a140cf710 tx=0x7f8a1002a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/387414024 shutdown_connections 2024-05-11T01:01:16.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/387414024 >> v2:172.21.15.119:3300/0 conn(0x7f8a140d1b50 0x7f8a140d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/387414024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a141602b0 0x7f8a141606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/387414024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a1415f430 0x7f8a1415fd70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/387414024 >> 172.21.15.52:0/387414024 conn(0x7f8a1408eb70 msgr2=0x7f8a1409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/387414024 shutdown_connections 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.398+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/387414024 wait complete. 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.400+0000 7f8a1c1f7640 1 Processor -- start 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.400+0000 7f8a1c1f7640 1 -- start start 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.400+0000 7f8a1c1f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140d1b50 0x7f8a140ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.400+0000 7f8a1c1f7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f430 0x7f8a141f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.400+0000 7f8a1c1f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a141602b0 0x7f8a140cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.400+0000 7f8a1c1f7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14163030 con 0x7f8a141602b0 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.400+0000 7f8a1c1f7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14162eb0 con 0x7f8a140d1b50 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.400+0000 7f8a1c1f7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a140d2540 con 0x7f8a1415f430 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1a794640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a141602b0 0x7f8a140cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1a794640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a141602b0 0x7f8a140cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35460/0 (socket says 172.21.15.52:35460) 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1a794640 1 -- 172.21.15.52:0/3493840881 learned_addr learned my addr 172.21.15.52:0/3493840881 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a19792640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f430 0x7f8a141f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a19f93640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140d1b50 0x7f8a140ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1a794640 1 -- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f430 msgr2=0x7f8a141f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1a794640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f430 0x7f8a141f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1a794640 1 -- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140d1b50 msgr2=0x7f8a140ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1a794640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140d1b50 0x7f8a140ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1a794640 1 -- 172.21.15.52:0/3493840881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1414af10 con 0x7f8a141602b0 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a19f93640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140d1b50 0x7f8a140ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1a794640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a141602b0 0x7f8a140cb4e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c007e60 tx=0x7f8a0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a0a7fc640 1 -- 172.21.15.52:0/3493840881 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c014620 con 0x7f8a141602b0 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a0a7fc640 1 -- 172.21.15.52:0/3493840881 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c019070 con 0x7f8a141602b0 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a141f9630 con 0x7f8a141602b0 2024-05-11T01:01:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a0a7fc640 1 -- 172.21.15.52:0/3493840881 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c014a20 con 0x7f8a141602b0 2024-05-11T01:01:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.401+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141476e0 con 0x7f8a141602b0 2024-05-11T01:01:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.402+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a14069780 con 0x7f8a141602b0 2024-05-11T01:01:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.403+0000 7f8a0a7fc640 1 -- 172.21.15.52:0/3493840881 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8a0c014bc0 con 0x7f8a141602b0 2024-05-11T01:01:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.403+0000 7f8a0a7fc640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89f8076300 0x7f89f80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.403+0000 7f8a0a7fc640 1 -- 172.21.15.52:0/3493840881 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a0c097d90 con 0x7f8a141602b0 2024-05-11T01:01:16.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.404+0000 7f8a19f93640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89f8076300 0x7f89f80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.404+0000 7f8a19f93640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89f8076300 0x7f89f80787c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a10034000 tx=0x7f8a1002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:16.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.407+0000 7f8a0a7fc640 1 -- 172.21.15.52:0/3493840881 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8a141f9630 con 0x7f8a141602b0 2024-05-11T01:01:16.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.601+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a140d2940 con 0x7f8a141602b0 2024-05-11T01:01:16.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.602+0000 7f8a0a7fc640 1 -- 172.21.15.52:0/3493840881 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f8a0c0616f0 con 0x7f8a141602b0 2024-05-11T01:01:16.603 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:16.603 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":20,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:02.673511+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:00:59.086890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:00.008780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:16.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89f8076300 msgr2=0x7f89f80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89f8076300 0x7f89f80787c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a10034000 tx=0x7f8a1002e000 comp rx=0 tx=0).stop 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a141602b0 msgr2=0x7f8a140cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a141602b0 0x7f8a140cb4e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c007e60 tx=0x7f8a0c00e4b0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 shutdown_connections 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89f8076300 0x7f89f80787c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a141602b0 0x7f8a140cb4e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f430 0x7f8a141f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 --2- 172.21.15.52:0/3493840881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140d1b50 0x7f8a140ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 >> 172.21.15.52:0/3493840881 conn(0x7f8a1408eb70 msgr2=0x7f8a140b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 shutdown_connections 2024-05-11T01:01:16.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.604+0000 7f8a1c1f7640 1 -- 172.21.15.52:0/3493840881 wait complete. 2024-05-11T01:01:16.617 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:16.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.721+0000 7f71c325c640 1 Processor -- start 2024-05-11T01:01:16.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.721+0000 7f71c325c640 1 -- start start 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c325c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f71bc15f430 0x7f71bc15fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c325c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71bc1602b0 0x7f71bc0d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c325c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71bc163250 con 0x7f71bc160720 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c325c640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f71bc163530 con 0x7f71bc15f430 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c325c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f71bc163810 con 0x7f71bc1602b0 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c0ff8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f71bc15f430 0x7f71bc15fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c0ff8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f71bc15f430 0x7f71bc15fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35476/0 (socket says 172.21.15.52:35476) 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c0ff8640 1 -- 172.21.15.52:0/1918271169 learned_addr learned my addr 172.21.15.52:0/1918271169 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71bbfff640 1 --2- 172.21.15.52:0/1918271169 >> v2:172.21.15.119:3300/0 conn(0x7f71bc1602b0 0x7f71bc0d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c0ff8640 1 -- 172.21.15.52:0/1918271169 >> v2:172.21.15.119:3300/0 conn(0x7f71bc1602b0 msgr2=0x7f71bc0d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:16.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c0ff8640 1 --2- 172.21.15.52:0/1918271169 >> v2:172.21.15.119:3300/0 conn(0x7f71bc1602b0 0x7f71bc0d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c0ff8640 1 -- 172.21.15.52:0/1918271169 >> v1:172.21.15.119:6789/0 conn(0x7f71bc160720 legacy=0x7f71bc0d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c0ff8640 1 -- 172.21.15.52:0/1918271169 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc163af0 con 0x7f71bc15f430 2024-05-11T01:01:16.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.722+0000 7f71c0ff8640 1 --2- 172.21.15.52:0/1918271169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc15f430 0x7f71bc15fd70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0b4840 tx=0x7f71b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9602002ce2ad796c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:16.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.724+0000 7f71bb7fe640 1 -- 172.21.15.52:0/1918271169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0037070 con 0x7f71bc15f430 2024-05-11T01:01:16.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.724+0000 7f71bb7fe640 1 -- 172.21.15.52:0/1918271169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b0032440 con 0x7f71bc15f430 2024-05-11T01:01:16.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.724+0000 7f71bb7fe640 1 -- 172.21.15.52:0/1918271169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0032810 con 0x7f71bc15f430 2024-05-11T01:01:16.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.725+0000 7f71c325c640 1 -- 172.21.15.52:0/1918271169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc15f430 msgr2=0x7f71bc15fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.725+0000 7f71c325c640 1 --2- 172.21.15.52:0/1918271169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc15f430 0x7f71bc15fd70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0b4840 tx=0x7f71b002a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.725+0000 7f71c325c640 1 -- 172.21.15.52:0/1918271169 shutdown_connections 2024-05-11T01:01:16.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.725+0000 7f71c325c640 1 --2- 172.21.15.52:0/1918271169 >> v2:172.21.15.119:3300/0 conn(0x7f71bc1602b0 0x7f71bc0d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.725+0000 7f71c325c640 1 --2- 172.21.15.52:0/1918271169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc15f430 0x7f71bc15fd70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.725+0000 7f71c325c640 1 -- 172.21.15.52:0/1918271169 >> 172.21.15.52:0/1918271169 conn(0x7f71bc08eb70 msgr2=0x7f71bc09e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:16.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.725+0000 7f71c325c640 1 -- 172.21.15.52:0/1918271169 shutdown_connections 2024-05-11T01:01:16.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.725+0000 7f71c325c640 1 -- 172.21.15.52:0/1918271169 wait complete. 2024-05-11T01:01:16.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.726+0000 7f71c325c640 1 Processor -- start 2024-05-11T01:01:16.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.726+0000 7f71c325c640 1 -- start start 2024-05-11T01:01:16.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c325c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71bc15f430 0x7f71bc147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c325c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71bc1602b0 0x7f71bc0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c325c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc160720 0x7f71bc161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c325c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc163030 con 0x7f71bc160720 2024-05-11T01:01:16.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c325c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc162eb0 con 0x7f71bc15f430 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c325c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f71bc0d27a0 con 0x7f71bc1602b0 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c17f9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc160720 0x7f71bc161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c17f9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc160720 0x7f71bc161520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35480/0 (socket says 172.21.15.52:35480) 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c17f9640 1 -- 172.21.15.52:0/329928257 learned_addr learned my addr 172.21.15.52:0/329928257 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71c0ff8640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71bc15f430 0x7f71bc147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.727+0000 7f71bbfff640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71bc1602b0 0x7f71bc0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71c17f9640 1 -- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71bc1602b0 msgr2=0x7f71bc0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71c17f9640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71bc1602b0 0x7f71bc0b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71c17f9640 1 -- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71bc15f430 msgr2=0x7f71bc147ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71c17f9640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71bc15f430 0x7f71bc147ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71c17f9640 1 -- 172.21.15.52:0/329928257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc0b3690 con 0x7f71bc160720 2024-05-11T01:01:16.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71c17f9640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc160720 0x7f71bc161520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c700 tx=0x7f71b400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:16.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71b97fa640 1 -- 172.21.15.52:0/329928257 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4003b70 con 0x7f71bc160720 2024-05-11T01:01:16.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71b97fa640 1 -- 172.21.15.52:0/329928257 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b4003d10 con 0x7f71bc160720 2024-05-11T01:01:16.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc0b3970 con 0x7f71bc160720 2024-05-11T01:01:16.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71b97fa640 1 -- 172.21.15.52:0/329928257 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b400aa70 con 0x7f71bc160720 2024-05-11T01:01:16.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.728+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc0b3f30 con 0x7f71bc160720 2024-05-11T01:01:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.729+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71bc069780 con 0x7f71bc160720 2024-05-11T01:01:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.729+0000 7f71b97fa640 1 -- 172.21.15.52:0/329928257 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f71b400ac10 con 0x7f71bc160720 2024-05-11T01:01:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.730+0000 7f71b97fa640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71a00762d0 0x7f71a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.730+0000 7f71b97fa640 1 -- 172.21.15.52:0/329928257 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f71b40973c0 con 0x7f71bc160720 2024-05-11T01:01:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.730+0000 7f71c0ff8640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71a00762d0 0x7f71a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:16.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.731+0000 7f71c0ff8640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71a00762d0 0x7f71a0078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71b0034000 tx=0x7f71b002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:16.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.735+0000 7f71b97fa640 1 -- 172.21.15.52:0/329928257 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f71b40606d0 con 0x7f71bc160720 2024-05-11T01:01:16.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.964+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71bc0b5d20 con 0x7f71bc160720 2024-05-11T01:01:16.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.965+0000 7f71b97fa640 1 -- 172.21.15.52:0/329928257 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f71b40649d0 con 0x7f71bc160720 2024-05-11T01:01:16.965 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:16.965 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2245862597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2245862597},{"type":"v1","addr":"172.21.15.119:6837","nonce":2245862597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4286578298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4286578298},{"type":"v1","addr":"172.21.15.52:6835","nonce":4286578298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:01.052672+0000","modified":"2024-05-11T01:01:04.065799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/1484114523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1484114523},{"type":"v1","addr":"172.21.15.119:6836","nonce":1484114523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-11T01:01:16.965 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-05-11T01:01:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71a00762d0 msgr2=0x7f71a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71a00762d0 0x7f71a0078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71b0034000 tx=0x7f71b002e000 comp rx=0 tx=0).stop 2024-05-11T01:01:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc160720 msgr2=0x7f71bc161520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc160720 0x7f71bc161520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c700 tx=0x7f71b400cbc0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 shutdown_connections 2024-05-11T01:01:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71a00762d0 0x7f71a0078790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71bc160720 0x7f71bc161520 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71bc1602b0 0x7f71bc0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 --2- 172.21.15.52:0/329928257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71bc15f430 0x7f71bc147ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 >> 172.21.15.52:0/329928257 conn(0x7f71bc08eb70 msgr2=0x7f71bc160db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 shutdown_connections 2024-05-11T01:01:16.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:16.967+0000 7f71c325c640 1 -- 172.21.15.52:0/329928257 wait complete. 2024-05-11T01:01:17.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T01:01:01.052672+0000', 'modified': '2024-05-11T01:01:04.065799+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6836/1484114523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 1484114523}, {'type': 'v1', 'addr': '172.21.15.119:6836', 'nonce': 1484114523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4276, 'qdb_cluster': [4276]} 2024-05-11T01:01:17.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T01:01:17.382 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-11T01:01:17.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.492+0000 7fc7a9be4640 1 Processor -- start 2024-05-11T01:01:17.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.492+0000 7fc7a9be4640 1 -- start start 2024-05-11T01:01:17.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a9be4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7a415f430 0x7fc7a415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a9be4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc7a41602b0 0x7fc7a40d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a9be4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a4160720 0x7fc7a40d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a9be4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a4163250 con 0x7fc7a41602b0 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a9be4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a4163530 con 0x7fc7a415f430 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a9be4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a4163810 con 0x7fc7a4160720 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a2ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc7a41602b0 0x7fc7a40d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a2ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc7a41602b0 0x7fc7a40d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35488/0 (socket says 172.21.15.52:35488) 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/342952280 learned_addr learned my addr 172.21.15.52:0/342952280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/342952280 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a4160720 msgr2=0x7fc7a40d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/342952280 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a4160720 0x7fc7a40d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/342952280 >> v2:172.21.15.119:3300/0 conn(0x7fc7a415f430 0x7fc7a415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/342952280 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a4160720 0x7fc7a40d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/342952280 >> v2:172.21.15.119:3300/0 conn(0x7fc7a415f430 msgr2=0x7fc7a415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/342952280 >> v2:172.21.15.119:3300/0 conn(0x7fc7a415f430 0x7fc7a415fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.493+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/342952280 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a4163af0 con 0x7fc7a41602b0 2024-05-11T01:01:17.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.494+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/342952280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 0x7fc7a40d1b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40b6b80 tx=0x7fc79802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d944f579978f90f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.494+0000 7fc7a27fc640 1 -- 172.21.15.52:0/342952280 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798037070 con 0x7fc7a41602b0 2024-05-11T01:01:17.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.494+0000 7fc7a27fc640 1 -- 172.21.15.52:0/342952280 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc798032440 con 0x7fc7a41602b0 2024-05-11T01:01:17.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.494+0000 7fc7a27fc640 1 -- 172.21.15.52:0/342952280 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798032810 con 0x7fc7a41602b0 2024-05-11T01:01:17.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.495+0000 7fc7a9be4640 1 -- 172.21.15.52:0/342952280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 msgr2=0x7fc7a40d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.495+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/342952280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 0x7fc7a40d1b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40b6b80 tx=0x7fc79802a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.495+0000 7fc7a9be4640 1 -- 172.21.15.52:0/342952280 shutdown_connections 2024-05-11T01:01:17.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.495+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/342952280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 0x7fc7a40d1b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.495+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/342952280 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a4160720 0x7fc7a40d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.495+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/342952280 >> v2:172.21.15.119:3300/0 conn(0x7fc7a415f430 0x7fc7a415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.495+0000 7fc7a9be4640 1 -- 172.21.15.52:0/342952280 >> 172.21.15.52:0/342952280 conn(0x7fc7a408eb70 msgr2=0x7fc7a409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:17.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.495+0000 7fc7a9be4640 1 -- 172.21.15.52:0/342952280 shutdown_connections 2024-05-11T01:01:17.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.495+0000 7fc7a9be4640 1 -- 172.21.15.52:0/342952280 wait complete. 2024-05-11T01:01:17.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.496+0000 7fc7a9be4640 1 Processor -- start 2024-05-11T01:01:17.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.496+0000 7fc7a9be4640 1 -- start start 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a9be4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a415f430 0x7fc7a415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a9be4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 0x7fc7a40ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a9be4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a4160720 0x7fc7a40cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a9be4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a4163030 con 0x7fc7a41602b0 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a9be4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a4162eb0 con 0x7fc7a4160720 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a9be4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a40d25d0 con 0x7fc7a415f430 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 0x7fc7a40ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 0x7fc7a40ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35496/0 (socket says 172.21.15.52:35496) 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3110515697 learned_addr learned my addr 172.21.15.52:0/3110515697 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a4160720 0x7fc7a40cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a415f430 0x7fc7a415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a415f430 msgr2=0x7fc7a415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a415f430 0x7fc7a415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a4160720 msgr2=0x7fc7a40cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a4160720 0x7fc7a40cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/3110515697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a415cd80 con 0x7fc7a41602b0 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a415f430 0x7fc7a415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.497+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 0x7fc7a40ca850 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc79802a220 tx=0x7fc7980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.498+0000 7fc78bfff640 1 -- 172.21.15.52:0/3110515697 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798037040 con 0x7fc7a41602b0 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.498+0000 7fc78bfff640 1 -- 172.21.15.52:0/3110515697 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc798042070 con 0x7fc7a41602b0 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.498+0000 7fc7a9be4640 1 -- 172.21.15.52:0/3110515697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a415aa40 con 0x7fc7a41602b0 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.498+0000 7fc78bfff640 1 -- 172.21.15.52:0/3110515697 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798030bb0 con 0x7fc7a41602b0 2024-05-11T01:01:17.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.498+0000 7fc7a9be4640 1 -- 172.21.15.52:0/3110515697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a40cc590 con 0x7fc7a41602b0 2024-05-11T01:01:17.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.500+0000 7fc78bfff640 1 -- 172.21.15.52:0/3110515697 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc798047460 con 0x7fc7a41602b0 2024-05-11T01:01:17.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.500+0000 7fc78bfff640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7900762d0 0x7fc790078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.500+0000 7fc78bfff640 1 -- 172.21.15.52:0/3110515697 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7980b8190 con 0x7fc7a41602b0 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.500+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7900762d0 0x7fc790078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.501+0000 7fc7ac9dc740 1 Processor -- start 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.501+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7900762d0 0x7fc790078790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc79400aa50 tx=0x7fc794008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.501+0000 7fc7ac9dc740 1 -- start start 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dbff0 0x558b847de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847ded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847aec50 0x558b847b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b847188d0 con 0x558b847dc3d0 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558b846d1c10 con 0x558b847aec50 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x558b84714890 con 0x558b847dbff0 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847ded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a37fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dbff0 0x558b847de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847ded40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35504/0 (socket says 172.21.15.52:35504) 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/1614536933 learned_addr learned my addr 172.21.15.52:0/1614536933 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/1614536933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847aec50 0x558b847b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/1614536933 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dbff0 msgr2=0x558b847de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/1614536933 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dbff0 0x558b847de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/1614536933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847aec50 msgr2=0x558b847b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/1614536933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847aec50 0x558b847b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/1614536933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b846d95d0 con 0x558b847dc3d0 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/1614536933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847ded40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc79807da70 tx=0x7fc79807dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc917a997404b2e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7897fa640 1 -- 172.21.15.52:0/1614536933 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798081b80 con 0x558b847dc3d0 2024-05-11T01:01:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7897fa640 1 -- 172.21.15.52:0/1614536933 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc798081480 con 0x558b847dc3d0 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.502+0000 7fc7897fa640 1 -- 172.21.15.52:0/1614536933 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798085830 con 0x558b847dc3d0 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.503+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/1614536933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 msgr2=0x558b847ded40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.503+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/1614536933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847ded40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc79807da70 tx=0x7fc79807dc00 comp rx=0 tx=0).stop 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.503+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/1614536933 shutdown_connections 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.503+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/1614536933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847aec50 0x558b847b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.503+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/1614536933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847ded40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.503+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/1614536933 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dbff0 0x558b847de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.503+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/1614536933 >> 172.21.15.52:0/1614536933 conn(0x558b847b0ec0 msgr2=0x558b847f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.503+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/1614536933 shutdown_connections 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.503+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/1614536933 wait complete. 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.504+0000 7fc7ac9dc740 1 -- start start 2024-05-11T01:01:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.504+0000 7fc7ac9dc740 1 Processor -- start 2024-05-11T01:01:17.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.504+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b848004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.504+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847e1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.504+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847fc460 0x558b847fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.504+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b846d1c10 con 0x558b847dc3d0 2024-05-11T01:01:17.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.504+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558b846d5760 con 0x558b847dbff0 2024-05-11T01:01:17.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.504+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x558b847188d0 con 0x558b847fc460 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7ac9dc740 10 client.0 _ll_register_callbacks cb 0x558b8470ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847e1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847e1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35514/0 (socket says 172.21.15.52:35514) 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/260201889 learned_addr learned my addr 172.21.15.52:0/260201889 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847fc460 0x558b847fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b848004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847fc460 msgr2=0x558b847fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847fc460 0x558b847fc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 msgr2=0x558b848004d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b848004d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a2ffd640 1 -- 172.21.15.52:0/260201889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8442a2d0 con 0x558b847dc3d0 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847e1e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc798000c00 tx=0x7fc798008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65f7fce96046573 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7827fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-11T01:01:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7827fc640 1 -- 172.21.15.52:0/260201889 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798081b80 con 0x558b847dc3d0 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7827fc640 1 -- 172.21.15.52:0/260201889 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7980b6330 con 0x558b847dc3d0 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/260201889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc798081480 con 0x558b847dc3d0 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7827fc640 1 -- 172.21.15.52:0/260201889 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fc79808a250 con 0x558b847dc3d0 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7ac9dc740 10 client.4611 fetch_fsmap learned FSMap version 8 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/260201889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x558b844f46b0 con 0x558b847dc3d0 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.505+0000 7fc7827fc640 1 -- 172.21.15.52:0/260201889 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc798081480 con 0x558b847dc3d0 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7827fc640 1 -- 172.21.15.52:0/260201889 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fc79803a030 con 0x558b847dc3d0 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7ac9dc740 10 client.4611 fetch_fsmap finished waiting for FSMap version 8 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7ac9dc740 10 client.4611 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7ac9dc740 20 client.4611 populate_metadata read hostname 'smithi052' 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] conn(0x558b8480d330 0x558b8480d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7ac9dc740 4 client.4611 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/260201889 --> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b8469c760 con 0x558b8480d330 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] conn(0x558b8480d330 0x558b8480d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] conn(0x558b8480d330 0x558b8480d710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21705960286c759 server_cookie=c8be016b41a98854 in_seq=0 out_seq=0 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.506+0000 7fc7827fc640 10 client.4611 ms_handle_connect on v2:172.21.15.119:6834/1484114523 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7827fc640 1 -- 172.21.15.52:0/260201889 <== mds.0 v2:172.21.15.119:6834/1484114523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b8469c760 con 0x558b8480d330 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7827fc640 10 client.4611 handle_command_reply: tid=0 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 client.4611 shutdown 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/260201889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 msgr2=0x558b847e1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847e1e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc798000c00 tx=0x7fc798008070 comp rx=0 tx=0).stop 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/260201889 shutdown_connections 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] conn(0x558b8480d330 0x558b8480d710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847fc460 0x558b847fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847dc3d0 0x558b847e1e60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7827fc640 0 client.4611 ms_handle_reset on v2:172.21.15.119:6834/1484114523 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/260201889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b848004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/260201889 >> 172.21.15.52:0/260201889 conn(0x558b847aec50 msgr2=0x558b847adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/260201889 shutdown_connections 2024-05-11T01:01:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.507+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/260201889 wait complete. 2024-05-11T01:01:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.508+0000 7fc7ac9dc740 20 client.4611 trim_cache size 0 max 16384 2024-05-11T01:01:17.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.526+0000 7fc7ac9dc740 1 Processor -- start 2024-05-11T01:01:17.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.526+0000 7fc7ac9dc740 1 -- start start 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b847fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 0x558b8481d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a4200b70 con 0x558b847fc460 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a415bd70 con 0x558b847dbff0 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a40ce0e0 con 0x558b847dc3d0 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35528/0 (socket says 172.21.15.52:35528) 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a3fff640 1 -- 172.21.15.52:0/3998491576 learned_addr learned my addr 172.21.15.52:0/3998491576 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/3998491576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 0x558b8481d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a3fff640 1 -- 172.21.15.52:0/3998491576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 msgr2=0x558b8481d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/3998491576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 0x558b8481d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/3998491576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b847fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a3fff640 1 -- 172.21.15.52:0/3998491576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 msgr2=0x558b847fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/3998491576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b847fd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.527+0000 7fc7a3fff640 1 -- 172.21.15.52:0/3998491576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b847f7220 con 0x558b847fc460 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/3998491576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b3fd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc79c007ed0 tx=0x7fc79c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3755087c822591e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc788ff9640 1 -- 172.21.15.52:0/3998491576 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c020070 con 0x558b847fc460 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc788ff9640 1 -- 172.21.15.52:0/3998491576 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c0105e0 con 0x558b847fc460 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc788ff9640 1 -- 172.21.15.52:0/3998491576 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0107b0 con 0x558b847fc460 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3998491576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 msgr2=0x558b847b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3998491576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b3fd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc79c007ed0 tx=0x7fc79c012820 comp rx=0 tx=0).stop 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3998491576 shutdown_connections 2024-05-11T01:01:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3998491576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b3fd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3998491576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 0x558b8481d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3998491576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b847fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3998491576 >> 172.21.15.52:0/3998491576 conn(0x558b847aec50 msgr2=0x558b847fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:17.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3998491576 shutdown_connections 2024-05-11T01:01:17.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.528+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3998491576 wait complete. 2024-05-11T01:01:17.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.529+0000 7fc7ac9dc740 1 -- start start 2024-05-11T01:01:17.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.529+0000 7fc7ac9dc740 1 Processor -- start 2024-05-11T01:01:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b847fa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 0x558b847b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7ac9dc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b846d1c10 con 0x558b847fc460 2024-05-11T01:01:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558b846d5760 con 0x558b847dbff0 2024-05-11T01:01:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7ac9dc740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x558b847e1580 con 0x558b847dc3d0 2024-05-11T01:01:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7ac9dc740 10 client.0 _ll_register_callbacks cb 0x7fc79c007990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35542/0 (socket says 172.21.15.52:35542) 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a3fff640 1 -- 172.21.15.52:0/3195286092 learned_addr learned my addr 172.21.15.52:0/3195286092 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b847fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a2ffd640 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 0x558b847b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a3fff640 1 -- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 msgr2=0x558b847b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 0x558b847b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a3fff640 1 -- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 msgr2=0x558b847fa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b847fa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a3fff640 1 -- 172.21.15.52:0/3195286092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b847fd1f0 con 0x558b847fc460 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b847fa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7a3fff640 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b2e30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc79c01e040 tx=0x7fc79c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c5b4a32c990adb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.530+0000 7fc7827fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-11T01:01:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7827fc640 1 -- 172.21.15.52:0/3195286092 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c020040 con 0x558b847fc460 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7827fc640 1 -- 172.21.15.52:0/3195286092 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc79c010790 con 0x558b847fc460 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3195286092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc79c0105e0 con 0x558b847fc460 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7827fc640 1 -- 172.21.15.52:0/3195286092 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fc79c031050 con 0x558b847fc460 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7ac9dc740 10 client.4617 fetch_fsmap learned FSMap version 8 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3195286092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x558b8481da10 con 0x558b847fc460 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7827fc640 1 -- 172.21.15.52:0/3195286092 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c0105e0 con 0x558b847fc460 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7827fc640 1 -- 172.21.15.52:0/3195286092 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fc79c034020 con 0x558b847fc460 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7ac9dc740 10 client.4617 fetch_fsmap finished waiting for FSMap version 8 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7ac9dc740 10 client.4617 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7ac9dc740 20 client.4617 populate_metadata read hostname 'smithi052' 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] conn(0x558b847d7960 0x558b847d7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7ac9dc740 4 client.4617 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3195286092 --> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] -- command(tid 0: {"prefix": "status"}) -- 0x558b847264e0 con 0x558b847d7960 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.531+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] conn(0x558b847d7960 0x558b847d7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7a37fe640 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] conn(0x558b847d7960 0x558b847d7d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=134b7ab2b39b6630 server_cookie=70c507bb6beaea9e in_seq=0 out_seq=0 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7827fc640 10 client.4617 ms_handle_connect on v2:172.21.15.119:6834/1484114523 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7827fc640 1 -- 172.21.15.52:0/3195286092 <== mds.0 v2:172.21.15.119:6834/1484114523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x558b847264e0 con 0x558b847d7960 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7827fc640 10 client.4617 handle_command_reply: tid=0 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 client.4617 shutdown 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3195286092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 msgr2=0x558b847b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b2e30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc79c01e040 tx=0x7fc79c004ab0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3195286092 shutdown_connections 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:6834/1484114523,v1:172.21.15.119:6836/1484114523] conn(0x558b847d7960 0x558b847d7d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b847fc460 0x558b847b2e30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x558b847dc3d0 0x558b847b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 --2- 172.21.15.52:0/3195286092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b847dbff0 0x558b847fa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7827fc640 0 client.4617 ms_handle_reset on v2:172.21.15.119:6834/1484114523 2024-05-11T01:01:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3195286092 >> 172.21.15.52:0/3195286092 conn(0x558b847aec50 msgr2=0x558b847b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3195286092 shutdown_connections 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 1 -- 172.21.15.52:0/3195286092 wait complete. 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.532+0000 7fc7ac9dc740 20 client.4617 trim_cache size 0 max 16384 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "f8411cb1-0508-49cc-906f-2364b6835afe", 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4276, 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-05-11T01:01:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 15.844401324, 2024-05-11T01:01:17.540 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 8, 2024-05-11T01:01:17.540 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 20, 2024-05-11T01:01:17.540 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 20, 2024-05-11T01:01:17.540 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 21.731554189000001 2024-05-11T01:01:17.540 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 -- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7900762d0 msgr2=0x7fc790078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7900762d0 0x7fc790078790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc79400aa50 tx=0x7fc794008040 comp rx=0 tx=0).stop 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 -- 172.21.15.52:0/3110515697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 msgr2=0x7fc7a40ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 0x7fc7a40ca850 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc79802a220 tx=0x7fc7980046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 -- 172.21.15.52:0/3110515697 shutdown_connections 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc7900762d0 0x7fc790078790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7a4160720 0x7fc7a40cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7a41602b0 0x7fc7a40ca850 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 --2- 172.21.15.52:0/3110515697 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7a415f430 0x7fc7a415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 -- 172.21.15.52:0/3110515697 >> 172.21.15.52:0/3110515697 conn(0x7fc7a408eb70 msgr2=0x7fc7a40d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 -- 172.21.15.52:0/3110515697 shutdown_connections 2024-05-11T01:01:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.533+0000 7fc7a9be4640 1 -- 172.21.15.52:0/3110515697 wait complete. 2024-05-11T01:01:17.544 INFO:teuthology.run_tasks:Running task sequential... 2024-05-11T01:01:17.553 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-11T01:01:17.553 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-11T01:01:17.554 INFO:teuthology.task.print:Enabling mgr modules 2024-05-11T01:01:17.554 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-11T01:01:17.612 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-11T01:01:17.612 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-11T01:01:17.612 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-11T01:01:17.612 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T01:01:17.612 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:17.613 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-11T01:01:17.613 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T01:01:17.613 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-11T01:01:17.613 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-11T01:01:17.613 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T01:01:17.613 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:17.613 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-11T01:01:17.613 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T01:01:17.613 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-11T01:01:17.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:17.613 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-05-11T01:01:17.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:17.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-05-11T01:01:17.705 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2024-05-11T01:01:17.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T01:01:17.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:17.707 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-11T01:01:17.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:17.724 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-05-11T01:01:17.792 INFO:teuthology.orchestra.run.smithi052.stderr:Cannot find device "ceph-brx" 2024-05-11T01:01:17.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T01:01:17.794 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-11T01:01:17.794 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T01:01:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.938+0000 7f45a24a7640 1 Processor -- start 2024-05-11T01:01:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.939+0000 7f45a24a7640 1 -- start start 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.939+0000 7f45a24a7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f459c15d180 0x7f459c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.939+0000 7f45a24a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c0d1450 0x7f459c0d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.939+0000 7f45a24a7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f459c0b04e0 con 0x7f459c0b1580 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.939+0000 7f45a24a7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f459c0d1df0 con 0x7f459c15d180 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.939+0000 7f45a24a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f459c0b10f0 con 0x7f459c0d1450 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f459c15d180 0x7f459c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c0d1450 0x7f459c0d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f459c15d180 0x7f459c0afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:46876/0 (socket says 172.21.15.52:46876) 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c0d1450 0x7f459c0d18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41492/0 (socket says 172.21.15.52:41492) 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f45a0a44640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f459c0b1580 0x7f459c0b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:53042/0 (socket says 172.21.15.52:53042) 2024-05-11T01:01:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459bfff640 1 -- 172.21.15.52:0/1692865320 learned_addr learned my addr 172.21.15.52:0/1692865320 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459b7fe640 1 -- 172.21.15.52:0/1692865320 >> v2:172.21.15.119:3300/0 conn(0x7f459c15d180 msgr2=0x7f459c0afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459b7fe640 1 --2- 172.21.15.52:0/1692865320 >> v2:172.21.15.119:3300/0 conn(0x7f459c15d180 0x7f459c0afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459b7fe640 1 -- 172.21.15.52:0/1692865320 >> v1:172.21.15.119:6789/0 conn(0x7f459c0b1580 legacy=0x7f459c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459affd640 1 -- 172.21.15.52:0/1692865320 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1806809441 0 0) 0x7f459c0b04e0 con 0x7f459c0b1580 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459b7fe640 1 -- 172.21.15.52:0/1692865320 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c0b07c0 con 0x7f459c0d1450 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.940+0000 7f459bfff640 1 --2- 172.21.15.52:0/1692865320 >> v2:172.21.15.119:3300/0 conn(0x7f459c15d180 0x7f459c0afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.941+0000 7f459b7fe640 1 --2- 172.21.15.52:0/1692865320 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c0d1450 0x7f459c0d18b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f45900097a0 tx=0x7f459002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611b68b918849c54 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.941+0000 7f459affd640 1 -- 172.21.15.52:0/1692865320 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590037070 con 0x7f459c0d1450 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.941+0000 7f459affd640 1 -- 172.21.15.52:0/1692865320 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4590032440 con 0x7f459c0d1450 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.941+0000 7f459affd640 1 -- 172.21.15.52:0/1692865320 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590032810 con 0x7f459c0d1450 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.942+0000 7f45a24a7640 1 -- 172.21.15.52:0/1692865320 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c0d1450 msgr2=0x7f459c0d18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.942+0000 7f45a24a7640 1 --2- 172.21.15.52:0/1692865320 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c0d1450 0x7f459c0d18b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f45900097a0 tx=0x7f459002a180 comp rx=0 tx=0).stop 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.942+0000 7f45a24a7640 1 -- 172.21.15.52:0/1692865320 shutdown_connections 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.942+0000 7f45a24a7640 1 --2- 172.21.15.52:0/1692865320 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c0d1450 0x7f459c0d18b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.942+0000 7f45a24a7640 1 --2- 172.21.15.52:0/1692865320 >> v2:172.21.15.119:3300/0 conn(0x7f459c15d180 0x7f459c0afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.942+0000 7f45a24a7640 1 -- 172.21.15.52:0/1692865320 >> 172.21.15.52:0/1692865320 conn(0x7f459c09ba20 msgr2=0x7f459c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:17.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.942+0000 7f45a24a7640 1 -- 172.21.15.52:0/1692865320 shutdown_connections 2024-05-11T01:01:17.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.942+0000 7f45a24a7640 1 -- 172.21.15.52:0/1692865320 wait complete. 2024-05-11T01:01:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.943+0000 7f45a24a7640 1 Processor -- start 2024-05-11T01:01:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.943+0000 7f45a24a7640 1 -- start start 2024-05-11T01:01:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f45a24a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f459c0b1580 0x7f459c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f45a24a7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c0d1450 0x7f459c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f45a24a7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c15d180 0x7f459c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f45a24a7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f459c15d560 con 0x7f459c0d1450 2024-05-11T01:01:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f45a24a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0b0d80 con 0x7f459c0b1580 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f45a24a7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f459c0b01d0 con 0x7f459c15d180 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c0d1450 0x7f459c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c0d1450 0x7f459c15d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35548/0 (socket says 172.21.15.52:35548) 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459b7fe640 1 -- 172.21.15.52:0/2758618975 learned_addr learned my addr 172.21.15.52:0/2758618975 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459bfff640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f459c0b1580 0x7f459c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f45a0a44640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c15d180 0x7f459c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459b7fe640 1 -- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c15d180 msgr2=0x7f459c06e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459b7fe640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c15d180 0x7f459c06e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459b7fe640 1 -- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f459c0b1580 msgr2=0x7f459c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459b7fe640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f459c0b1580 0x7f459c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459b7fe640 1 -- 172.21.15.52:0/2758618975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c15dcb0 con 0x7f459c0d1450 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.944+0000 7f459b7fe640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c0d1450 0x7f459c15d770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f45900097a0 tx=0x7f45900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.945+0000 7f4598ff9640 1 -- 172.21.15.52:0/2758618975 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590037050 con 0x7f459c0d1450 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.945+0000 7f4598ff9640 1 -- 172.21.15.52:0/2758618975 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4590038040 con 0x7f459c0d1450 2024-05-11T01:01:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.945+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459c15df30 con 0x7f459c0d1450 2024-05-11T01:01:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.945+0000 7f4598ff9640 1 -- 172.21.15.52:0/2758618975 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590004490 con 0x7f459c0d1450 2024-05-11T01:01:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.945+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459c0b68c0 con 0x7f459c0d1450 2024-05-11T01:01:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.946+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459c15e5b0 con 0x7f459c0d1450 2024-05-11T01:01:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.946+0000 7f4598ff9640 1 -- 172.21.15.52:0/2758618975 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f459003c6d0 con 0x7f459c0d1450 2024-05-11T01:01:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.947+0000 7f4598ff9640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f45880762d0 0x7f4588078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.947+0000 7f4598ff9640 1 -- 172.21.15.52:0/2758618975 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f45900b7e10 con 0x7f459c0d1450 2024-05-11T01:01:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.947+0000 7f459bfff640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f45880762d0 0x7f4588078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.948+0000 7f459bfff640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f45880762d0 0x7f4588078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f458c006ca0 tx=0x7f458c009240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:17.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:17.951+0000 7f4598ff9640 1 -- 172.21.15.52:0/2758618975 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f459c15dcb0 con 0x7f459c0d1450 2024-05-11T01:01:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.181+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f459c0b3690 con 0x7f459c0d1450 2024-05-11T01:01:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.181+0000 7f4598ff9640 1 -- 172.21.15.52:0/2758618975 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f45900811b0 con 0x7f459c0d1450 2024-05-11T01:01:18.182 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T01:01:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f45880762d0 msgr2=0x7f4588078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f45880762d0 0x7f4588078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f458c006ca0 tx=0x7f458c009240 comp rx=0 tx=0).stop 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c0d1450 msgr2=0x7f459c15d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c0d1450 0x7f459c15d770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f45900097a0 tx=0x7f45900046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 shutdown_connections 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f45880762d0 0x7f4588078790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f459c15d180 0x7f459c06e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c0d1450 0x7f459c15d770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 --2- 172.21.15.52:0/2758618975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f459c0b1580 0x7f459c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 >> 172.21.15.52:0/2758618975 conn(0x7f459c09ba20 msgr2=0x7f459c071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 shutdown_connections 2024-05-11T01:01:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18.183+0000 7f45a24a7640 1 -- 172.21.15.52:0/2758618975 wait complete. 2024-05-11T01:01:18.195 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T01:01:18.195 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T01:01:18.195 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-05-11T01:01:18.195 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T01:01:18.195 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:18.196 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T01:01:18.196 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T01:01:18.196 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T01:01:18.196 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T01:01:18.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:18.196 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-05-11T01:01:18.213 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:01:18.213 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:01:18.213 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:01:18.213 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:18.213 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-11T01:01:18.213 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:18.213 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:18.213 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:18.213 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42763sec preferred_lft 42763sec 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:18.214 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-05-11T01:01:18.215 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T01:01:18.215 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:18.215 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:18.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-05-11T01:01:18.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-05-11T01:01:18.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-05-11T01:01:18.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T01:01:18.215 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:18.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:18.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:18.409 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T01:01:18.436 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-05-11T01:01:18.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:18.438 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-05-11T01:01:18.509 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-05-11T01:01:18.509 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T01:01:18.509 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-11T01:01:18.509 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-11T01:01:18.509 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T01:01:18.510 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:18.510 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:18.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-11T01:01:18.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-11T01:01:18.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-11T01:01:18.510 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:18.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:18.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:18.788 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-11T01:01:18.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:18.805 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-05-11T01:01:18.862 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:18.862 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:18.862 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:18.862 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-11T01:01:18.862 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:18.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:18.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:18.985 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-11T01:01:18.985 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:18.985 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:18.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-11T01:01:18.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T01:01:18.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-11T01:01:18.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-11T01:01:18.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-11T01:01:18.985 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:19.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:19.122 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:19.122 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:19.123 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-05-11T01:01:19.123 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T01:01:19.123 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:19.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:19.203 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-11T01:01:19.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:19.203 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:19.219 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T01:01:19.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:19.220 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:19.279 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-11T01:01:19.346 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-11T01:01:19.407 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:19.408 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-11T01:01:19.409 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-11T01:01:19.409 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T01:01:19.409 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T01:01:19.409 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T01:01:19.409 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T01:01:19.409 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-11T01:01:19.409 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:19.409 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:19.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:19.409 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-11T01:01:19.463 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-11T01:01:19.463 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-11T01:01:19.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-11T01:01:19.531 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:19.589 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-11T01:01:19.589 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-11T01:01:19.589 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-11T01:01:19.589 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-11T01:01:19.589 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-11T01:01:19.589 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-11T01:01:19.589 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-11T01:01:19.589 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-11T01:01:19.590 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-11T01:01:19.591 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:19.592 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:19.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:19.592 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-11T01:01:19.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:2024-05-11T01:01:19.626+0000 7f54d921f480 -1 init, newargv = 0x559f4bf79fe0 newargc=15 2024-05-11T01:01:19.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[43749]: starting ceph client 2024-05-11T01:01:19.631 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[43749]: starting fuse 2024-05-11T01:01:19.644 INFO:teuthology.orchestra.run.smithi052.stdout:40 2024-05-11T01:01:19.644 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-11T01:01:19.644 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-05-11T01:01:19.645 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.0")) 2024-05-11T01:01:19.646 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-05-11T01:01:19.769 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.0.43749.asok 2024-05-11T01:01:19.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:19.780 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43749.asok 2024-05-11T01:01:19.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:19.781 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43749.asok status 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "10380055a1fc5e7d3b37b252af7045f5c2c8ba29", 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-3570-g10380055 (10380055a1fc5e7d3b37b252af7045f5c2c8ba29) squid (dev)", 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "0", 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "43749", 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-05-11T01:01:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4629, 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 4629 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-11T01:01:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2479066651 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2479066651 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.4629 192.168.0.1:0/2479066651", 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/2479066651", 2024-05-11T01:01:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-05-11T01:01:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 8, 2024-05-11T01:01:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 20, 2024-05-11T01:01:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-05-11T01:01:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-05-11T01:01:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-05-11T01:01:19.924 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:01:19.930 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T01:01:20.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.034+0000 7f208ea93640 1 Processor -- start 2024-05-11T01:01:20.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.034+0000 7f208ea93640 1 -- start start 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f208ea93640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20880d15b0 0x7f20880cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f208ea93640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20880d0510 con 0x7f208815f530 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f208ea93640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20880d07f0 con 0x7f20880d3eb0 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f208ea93640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f20880d0ad0 con 0x7f20880d15b0 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f208c82f640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f208815f530 0x7f20880d37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:58184/0 (socket says 172.21.15.52:58184) 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f208c82f640 1 -- 172.21.15.52:0/2399787185 learned_addr learned my addr 172.21.15.52:0/2399787185 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f208d030640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20880d15b0 0x7f20880cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2650219155 0 0) 0x7f20880d0510 con 0x7f208815f530 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2078003520 con 0x7f208815f530 2024-05-11T01:01:20.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3716545857 0 0) 0x7f20880d07f0 con 0x7f20880d3eb0 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20880d0510 con 0x7f20880d3eb0 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2328700159 0 0) 0x7f2078003520 con 0x7f208815f530 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20880d15b0 msgr2=0x7f20880cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f20877fe640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20880d15b0 0x7f20880cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 >> v1:172.21.15.119:6789/0 conn(0x7f20880d3eb0 legacy=0x7f20880c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.035+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20880d0db0 con 0x7f208815f530 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f207c0016f0 con 0x7f208815f530 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 >> v1:172.21.15.52:6789/0 conn(0x7f208815f530 legacy=0x7f20880d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f20877fe640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20880d15b0 0x7f2078003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f20877fe640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078004210 0x7f2078004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f20877fe640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2078004d90 0x7f2078005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f20880d0ad0 con 0x7f2078004d90 2024-05-11T01:01:20.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f207800bbd0 con 0x7f2078004210 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f207800bdd0 con 0x7f20880d15b0 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f208d030640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2078004d90 0x7f2078005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f208c82f640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20880d15b0 0x7f2078003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f2087fff640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078004210 0x7f2078004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f208d030640 1 -- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20880d15b0 msgr2=0x7f2078003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f208d030640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20880d15b0 0x7f2078003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f208d030640 1 -- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078004210 msgr2=0x7f2078004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f208d030640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078004210 0x7f2078004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.036+0000 7f208d030640 1 -- 172.21.15.52:0/2399787185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2078003520 con 0x7f2078004d90 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f208d030640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2078004d90 0x7f2078005210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2080009f90 tx=0x7f2078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f721fdd7d4bcb812 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080009c10 con 0x7f2078004d90 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2080036070 con 0x7f2078004d90 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f20877fe640 1 -- 172.21.15.52:0/2399787185 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080031810 con 0x7f2078004d90 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f208ea93640 1 -- 172.21.15.52:0/2399787185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2078004d90 msgr2=0x7f2078005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f208ea93640 1 --2- 172.21.15.52:0/2399787185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2078004d90 0x7f2078005210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2080009f90 tx=0x7f2078005830 comp rx=0 tx=0).stop 2024-05-11T01:01:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f208d030640 1 -- 172.21.15.52:0/2399787185 reap_dead start 2024-05-11T01:01:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f208ea93640 1 -- 172.21.15.52:0/2399787185 shutdown_connections 2024-05-11T01:01:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f208ea93640 1 -- 172.21.15.52:0/2399787185 >> 172.21.15.52:0/2399787185 conn(0x7f208808cfb0 msgr2=0x7f208809e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.037+0000 7f208ea93640 1 -- 172.21.15.52:0/2399787185 shutdown_connections 2024-05-11T01:01:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.038+0000 7f208ea93640 1 -- 172.21.15.52:0/2399787185 wait complete. 2024-05-11T01:01:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.039+0000 7f208ea93640 1 Processor -- start 2024-05-11T01:01:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.039+0000 7f208ea93640 1 -- start start 2024-05-11T01:01:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208ea93640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2078004d90 0x7f208814f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208ea93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20881f53b0 0x7f20880b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208ea93640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20880b5770 0x7f208815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208ea93640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f208815fa90 con 0x7f20880b5770 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208ea93640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f208815f910 con 0x7f20881f53b0 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208ea93640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f20880cfe30 con 0x7f2078004d90 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208d030640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20880b5770 0x7f208815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208d030640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20880b5770 0x7f208815af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35552/0 (socket says 172.21.15.52:35552) 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208d030640 1 -- 172.21.15.52:0/1787134943 learned_addr learned my addr 172.21.15.52:0/1787134943 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f2087fff640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20881f53b0 0x7f20880b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208c82f640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2078004d90 0x7f208814f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208d030640 1 -- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2078004d90 msgr2=0x7f208814f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208d030640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2078004d90 0x7f208814f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208d030640 1 -- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20881f53b0 msgr2=0x7f20880b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208d030640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20881f53b0 0x7f20880b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208d030640 1 -- 172.21.15.52:0/1787134943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20880b2050 con 0x7f20880b5770 2024-05-11T01:01:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208d030640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20880b5770 0x7f208815af50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2080009f90 tx=0x7f2080009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f20857fa640 1 -- 172.21.15.52:0/1787134943 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080009c10 con 0x7f20880b5770 2024-05-11T01:01:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f20857fa640 1 -- 172.21.15.52:0/1787134943 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2080045430 con 0x7f20880b5770 2024-05-11T01:01:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f20857fa640 1 -- 172.21.15.52:0/1787134943 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080045800 con 0x7f20880b5770 2024-05-11T01:01:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.040+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20880b2330 con 0x7f20880b5770 2024-05-11T01:01:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.041+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20880b28f0 con 0x7f20880b5770 2024-05-11T01:01:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.041+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20880b2af0 con 0x7f20880b5770 2024-05-11T01:01:20.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.042+0000 7f20857fa640 1 -- 172.21.15.52:0/1787134943 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2080005d10 con 0x7f20880b5770 2024-05-11T01:01:20.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.043+0000 7f20857fa640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f206c0762d0 0x7f206c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:20.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.043+0000 7f20857fa640 1 -- 172.21.15.52:0/1787134943 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f20800b83c0 con 0x7f20880b5770 2024-05-11T01:01:20.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.043+0000 7f208c82f640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f206c0762d0 0x7f206c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:20.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.043+0000 7f208c82f640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f206c0762d0 0x7f206c078790 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f207c001220 tx=0x7f207c0011b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:20.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.047+0000 7f20857fa640 1 -- 172.21.15.52:0/1787134943 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f20880b2af0 con 0x7f20880b5770 2024-05-11T01:01:20.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.279+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f208815fda0 con 0x7f20880b5770 2024-05-11T01:01:20.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.279+0000 7f20857fa640 1 -- 172.21.15.52:0/1787134943 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f208002e050 con 0x7f20880b5770 2024-05-11T01:01:20.279 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T01:01:20.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f206c0762d0 msgr2=0x7f206c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:20.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f206c0762d0 0x7f206c078790 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f207c001220 tx=0x7f207c0011b0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20880b5770 msgr2=0x7f208815af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:20.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20880b5770 0x7f208815af50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2080009f90 tx=0x7f2080009750 comp rx=0 tx=0).stop 2024-05-11T01:01:20.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 shutdown_connections 2024-05-11T01:01:20.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f206c0762d0 0x7f206c078790 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20880b5770 0x7f208815af50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20881f53b0 0x7f20880b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 --2- 172.21.15.52:0/1787134943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2078004d90 0x7f208814f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:20.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 >> 172.21.15.52:0/1787134943 conn(0x7f208808cfb0 msgr2=0x7f208815e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:20.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 shutdown_connections 2024-05-11T01:01:20.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20.281+0000 7f208ea93640 1 -- 172.21.15.52:0/1787134943 wait complete. 2024-05-11T01:01:20.292 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T01:01:20.292 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T01:01:20.292 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-05-11T01:01:20.292 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-11T01:01:20.292 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-11T01:01:20.292 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T01:01:20.292 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T01:01:20.292 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T01:01:20.292 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-11T01:01:20.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:20.293 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-05-11T01:01:20.309 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:01:20.309 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:01:20.309 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-11T01:01:20.310 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42761sec preferred_lft 42761sec 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 1e:e6:90:cd:c4:f4 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::8c71:f4ff:fe68:49bf/64 scope link 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 1e:e6:90:cd:c4:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::1ce6:90ff:fecd:c4f4/64 scope link tentative 2024-05-11T01:01:20.311 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:20.313 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T01:01:20.374 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-05-11T01:01:20.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:20.376 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-05-11T01:01:20.433 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-05-11T01:01:20.433 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T01:01:20.433 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-11T01:01:20.433 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-11T01:01:20.433 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T01:01:20.433 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:20.434 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:20.434 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-11T01:01:20.434 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-11T01:01:20.434 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-11T01:01:20.434 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:20.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:20.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:20.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:20.564 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-11T01:01:20.581 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T01:01:20.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:20.582 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-05-11T01:01:20.637 INFO:teuthology.orchestra.run.smithi052.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-11T01:01:20.638 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:20.638 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:20.638 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:20.638 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-11T01:01:20.638 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:20.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:20.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:20.759 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-11T01:01:20.784 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 96:cd:d3:71:9c:96 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::94cd:d3ff:fe71:9c96/64 scope link 2024-05-11T01:01:20.785 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:20.786 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 96:cd:d3:71:9c:96 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::94cd:d3ff:fe71:9c96/64 scope link 2024-05-11T01:01:20.853 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:20.855 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-11T01:01:20.855 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:20.855 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:20.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-11T01:01:20.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-11T01:01:20.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-11T01:01:20.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-11T01:01:20.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-11T01:01:20.855 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:21.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:21.012 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:21.012 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:21.012 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.1 up 2024-05-11T01:01:21.012 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.1 master ceph-brx 2024-05-11T01:01:21.012 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:21.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:21.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:21.091 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-11T01:01:21.091 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:21.091 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-11T01:01:21.107 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-11T01:01:21.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:21.107 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-11T01:01:21.163 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-11T01:01:21.230 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:21.288 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-11T01:01:21.289 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-11T01:01:21.289 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-11T01:01:21.289 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-11T01:01:21.289 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-11T01:01:21.289 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-11T01:01:21.289 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-11T01:01:21.289 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:21.290 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:21.292 INFO:teuthology.orchestra.run.smithi052.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:21.292 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:21.292 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:21.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:21.292 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-11T01:01:21.345 INFO:teuthology.orchestra.run.smithi052.stdout:40 2024-05-11T01:01:21.345 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-05-11T01:01:21.345 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-11T01:01:21.389 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-11T01:01:21.415 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:21.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi052.stderr:2024-05-11T01:01:21.472+0000 7fe9bdd1e480 -1 init, newargv = 0x560c03a221e0 newargc=15 2024-05-11T01:01:21.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi052.stderr:ceph-fuse[44230]: starting ceph client 2024-05-11T01:01:21.473 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-11T01:01:21.473 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-11T01:01:21.473 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-11T01:01:21.473 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-11T01:01:21.473 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-11T01:01:21.473 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-11T01:01:21.473 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-11T01:01:21.473 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:21.474 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-11T01:01:21.475 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:21.476 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:21.476 INFO:teuthology.orchestra.run.smithi052.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:21.476 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:21.476 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:21.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:21.477 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-11T01:01:21.478 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi052.stderr:ceph-fuse[44230]: starting fuse 2024-05-11T01:01:21.533 INFO:teuthology.orchestra.run.smithi052.stdout:40 2024-05-11T01:01:21.533 INFO:teuthology.orchestra.run.smithi052.stdout:43 2024-05-11T01:01:21.533 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 43] 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-05-11T01:01:21.534 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.2")) 2024-05-11T01:01:21.535 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-05-11T01:01:21.655 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.2.44230.asok 2024-05-11T01:01:21.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:21.666 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.44230.asok 2024-05-11T01:01:21.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:21.666 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.44230.asok status 2024-05-11T01:01:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:01:21.804 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-05-11T01:01:21.804 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "10380055a1fc5e7d3b37b252af7045f5c2c8ba29", 2024-05-11T01:01:21.804 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-3570-g10380055 (10380055a1fc5e7d3b37b252af7045f5c2c8ba29) squid (dev)", 2024-05-11T01:01:21.804 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "2", 2024-05-11T01:01:21.804 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-05-11T01:01:21.804 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-11T01:01:21.804 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "44230", 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4641, 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 4641 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.2:0", 2024-05-11T01:01:21.805 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1792833643 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.2:0", 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1792833643 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.4641 192.168.0.2:0/1792833643", 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.2:0/1792833643", 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 8, 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 20, 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-05-11T01:01:21.806 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-05-11T01:01:21.807 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-05-11T01:01:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:01:21.813 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T01:01:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.916+0000 7f235af9d640 1 Processor -- start 2024-05-11T01:01:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.916+0000 7f235af9d640 1 -- start start 2024-05-11T01:01:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.917+0000 7f235af9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f235415f6e0 0x7f23540b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.917+0000 7f235af9d640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23540d26c0 con 0x7f23540d2220 2024-05-11T01:01:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.917+0000 7f235af9d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23540d29a0 con 0x7f23540b54e0 2024-05-11T01:01:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.917+0000 7f235af9d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f235415fac0 con 0x7f235415f6e0 2024-05-11T01:01:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.917+0000 7f2353fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f23540d2220 0x7f23540d3b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:59602/0 (socket says 172.21.15.52:59602) 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.917+0000 7f2353fff640 1 -- 172.21.15.52:0/1602154476 learned_addr learned my addr 172.21.15.52:0/1602154476 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.917+0000 7f2358d39640 1 --2- 172.21.15.52:0/1602154476 >> v2:172.21.15.119:3300/0 conn(0x7f235415f6e0 0x7f23540b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.917+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3529887048 0 0) 0x7f23540d26c0 con 0x7f23540d2220 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.917+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f233c003520 con 0x7f23540d2220 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3254380410 0 0) 0x7f23540d29a0 con 0x7f23540b54e0 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23540d26c0 con 0x7f23540b54e0 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2105114584 0 0) 0x7f233c003520 con 0x7f23540d2220 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 >> v2:172.21.15.119:3300/0 conn(0x7f235415f6e0 msgr2=0x7f23540b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 --2- 172.21.15.52:0/1602154476 >> v2:172.21.15.119:3300/0 conn(0x7f235415f6e0 0x7f23540b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 >> v1:172.21.15.119:6789/0 conn(0x7f23540b54e0 legacy=0x7f23540d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23540d01c0 con 0x7f23540d2220 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f23480016f0 con 0x7f23540d2220 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 >> v1:172.21.15.52:6789/0 conn(0x7f23540d2220 legacy=0x7f23540d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233c003c30 0x7f233c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f233c004800 0x7f233c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f233c005210 0x7f233c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f235415fac0 con 0x7f233c005210 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f233c00c140 con 0x7f233c003c30 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f233c00c340 con 0x7f233c004800 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f235953a640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f233c005210 0x7f233c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f2358d39640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233c003c30 0x7f233c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f2353fff640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f233c004800 0x7f233c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f235953a640 1 -- 172.21.15.52:0/1602154476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f233c004800 msgr2=0x7f233c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f235953a640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f233c004800 0x7f233c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f235953a640 1 -- 172.21.15.52:0/1602154476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233c003c30 msgr2=0x7f233c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f235953a640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233c003c30 0x7f233c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.918+0000 7f235953a640 1 -- 172.21.15.52:0/1602154476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c003520 con 0x7f233c005210 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.919+0000 7f235953a640 1 -- 172.21.15.52:0/1602154476 reap_dead start 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.919+0000 7f235953a640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f233c005210 0x7f233c00b9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f234c001fd0 tx=0x7f233c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d58e065969cd5b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.919+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c001c20 con 0x7f233c005210 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.919+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c003b70 con 0x7f233c005210 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.919+0000 7f23537fe640 1 -- 172.21.15.52:0/1602154476 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c0050d0 con 0x7f233c005210 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.920+0000 7f235af9d640 1 -- 172.21.15.52:0/1602154476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f233c005210 msgr2=0x7f233c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.920+0000 7f235af9d640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f233c005210 0x7f233c00b9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f234c001fd0 tx=0x7f233c005830 comp rx=0 tx=0).stop 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.920+0000 7f235af9d640 1 -- 172.21.15.52:0/1602154476 shutdown_connections 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.920+0000 7f235af9d640 1 --2- 172.21.15.52:0/1602154476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f233c005210 0x7f233c00b9c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.920+0000 7f235af9d640 1 -- 172.21.15.52:0/1602154476 >> 172.21.15.52:0/1602154476 conn(0x7f235408ec20 msgr2=0x7f235409e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.920+0000 7f235af9d640 1 -- 172.21.15.52:0/1602154476 shutdown_connections 2024-05-11T01:01:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.920+0000 7f235af9d640 1 -- 172.21.15.52:0/1602154476 wait complete. 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.921+0000 7f235af9d640 1 Processor -- start 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.921+0000 7f235af9d640 1 -- start start 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235af9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f233c005210 0x7f235414c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235af9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23540cb7d0 0x7f23541f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235af9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23541f8c70 0x7f235414bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235af9d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f23540b4fa0 con 0x7f23541f8c70 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235af9d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23540d0530 con 0x7f23540cb7d0 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235af9d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f23540b5120 con 0x7f233c005210 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235953a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23541f8c70 0x7f235414bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235953a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23541f8c70 0x7f235414bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43368/0 (socket says 172.21.15.52:43368) 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235953a640 1 -- 172.21.15.52:0/2767224476 learned_addr learned my addr 172.21.15.52:0/2767224476 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:21.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f2358d39640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f233c005210 0x7f235414c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f2353fff640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23540cb7d0 0x7f23541f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235953a640 1 -- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f233c005210 msgr2=0x7f235414c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235953a640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f233c005210 0x7f235414c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235953a640 1 -- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23540cb7d0 msgr2=0x7f23541f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235953a640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23540cb7d0 0x7f23541f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235953a640 1 -- 172.21.15.52:0/2767224476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23540babf0 con 0x7f23541f8c70 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.922+0000 7f235953a640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23541f8c70 0x7f235414bd30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f234c001fd0 tx=0x7f234c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.923+0000 7f23517fa640 1 -- 172.21.15.52:0/2767224476 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c041070 con 0x7f23541f8c70 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.923+0000 7f23517fa640 1 -- 172.21.15.52:0/2767224476 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c035050 con 0x7f23541f8c70 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.923+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2354160e50 con 0x7f23541f8c70 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.923+0000 7f23517fa640 1 -- 172.21.15.52:0/2767224476 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c03c680 con 0x7f23541f8c70 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.923+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2354161410 con 0x7f23541f8c70 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.924+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2354161610 con 0x7f23541f8c70 2024-05-11T01:01:21.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.924+0000 7f23517fa640 1 -- 172.21.15.52:0/2767224476 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f234c044070 con 0x7f23541f8c70 2024-05-11T01:01:21.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.925+0000 7f23517fa640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f23400762d0 0x7f2340078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:21.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.925+0000 7f23517fa640 1 -- 172.21.15.52:0/2767224476 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f234c0b75b0 con 0x7f23541f8c70 2024-05-11T01:01:21.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.925+0000 7f2358d39640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f23400762d0 0x7f2340078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:21.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.926+0000 7f2358d39640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f23400762d0 0x7f2340078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2344009b40 tx=0x7f2344009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:21.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:21.929+0000 7f23517fa640 1 -- 172.21.15.52:0/2767224476 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2354161610 con 0x7f23541f8c70 2024-05-11T01:01:22.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.160+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f23540d60c0 con 0x7f23541f8c70 2024-05-11T01:01:22.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.161+0000 7f23517fa640 1 -- 172.21.15.52:0/2767224476 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f234c0808c0 con 0x7f23541f8c70 2024-05-11T01:01:22.161 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f23400762d0 msgr2=0x7f2340078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f23400762d0 0x7f2340078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2344009b40 tx=0x7f2344009190 comp rx=0 tx=0).stop 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23541f8c70 msgr2=0x7f235414bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23541f8c70 0x7f235414bd30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f234c001fd0 tx=0x7f234c001670 comp rx=0 tx=0).stop 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 shutdown_connections 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f23400762d0 0x7f2340078790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23541f8c70 0x7f235414bd30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23540cb7d0 0x7f23541f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 --2- 172.21.15.52:0/2767224476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f233c005210 0x7f235414c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:22.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 >> 172.21.15.52:0/2767224476 conn(0x7f235408ec20 msgr2=0x7f235406ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:22.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 shutdown_connections 2024-05-11T01:01:22.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:22.163+0000 7f235af9d640 1 -- 172.21.15.52:0/2767224476 wait complete. 2024-05-11T01:01:22.173 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T01:01:22.174 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T01:01:22.174 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-05-11T01:01:22.174 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T01:01:22.174 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:22.174 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T01:01:22.174 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T01:01:22.174 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T01:01:22.174 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T01:01:22.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:22.174 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-05-11T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-11T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42767sec preferred_lft 42767sec 2024-05-11T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-05-11T01:01:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:22.195 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T01:01:22.195 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:22.196 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-11T01:01:22.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2024-05-11T01:01:22.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2024-05-11T01:01:22.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2024-05-11T01:01:22.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T01:01:22.196 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-11T01:01:22.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:22.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:22.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:22.402 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T01:01:22.426 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-05-11T01:01:22.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:22.428 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-05-11T01:01:22.541 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-05-11T01:01:22.541 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T01:01:22.541 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T01:01:22.541 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T01:01:22.541 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T01:01:22.542 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:22.542 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-11T01:01:22.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T01:01:22.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T01:01:22.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T01:01:22.542 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-11T01:01:22.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:22.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:22.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:22.777 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-05-11T01:01:22.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:22.793 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2024-05-11T01:01:22.849 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:22.849 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-11T01:01:22.849 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:22.849 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-11T01:01:22.849 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-11T01:01:22.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:22.969 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-11T01:01:22.970 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:22.970 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-11T01:01:22.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-11T01:01:22.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T01:01:22.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-11T01:01:22.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-11T01:01:22.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-11T01:01:22.970 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-11T01:01:23.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:23.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:23.098 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:23.098 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-11T01:01:23.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-05-11T01:01:23.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T01:01:23.098 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-11T01:01:23.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:23.175 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-11T01:01:23.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:23.175 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:23.191 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T01:01:23.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:23.191 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:23.247 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2024-05-11T01:01:23.313 DEBUG:teuthology.orchestra.run.smithi119:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:23.370 INFO:teuthology.orchestra.run.smithi119.stdout:/proc 2024-05-11T01:01:23.370 INFO:teuthology.orchestra.run.smithi119.stdout:/sys 2024-05-11T01:01:23.370 INFO:teuthology.orchestra.run.smithi119.stdout:/dev 2024-05-11T01:01:23.370 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/security 2024-05-11T01:01:23.370 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/shm 2024-05-11T01:01:23.370 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/pts 2024-05-11T01:01:23.370 INFO:teuthology.orchestra.run.smithi119.stdout:/run 2024-05-11T01:01:23.370 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/pstore 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/bpf 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/ 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/selinux 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/hugepages 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/mqueue 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/debug 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/tracing 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:23.371 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/config 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/1000 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-4 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-5 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-6 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-7 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:23.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:23.373 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2024-05-11T01:01:23.425 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-11T01:01:23.425 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-11T01:01:23.467 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2024-05-11T01:01:23.491 DEBUG:teuthology.orchestra.run.smithi119:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:23.548 INFO:teuthology.orchestra.run.smithi119.stdout:/proc 2024-05-11T01:01:23.548 INFO:teuthology.orchestra.run.smithi119.stdout:/sys 2024-05-11T01:01:23.548 INFO:teuthology.orchestra.run.smithi119.stdout:/dev 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/security 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/shm 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/pts 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/run 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/pstore 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/bpf 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/ 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/selinux 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:23.549 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/hugepages 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/mqueue 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/debug 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/tracing 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/config 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/1000 2024-05-11T01:01:23.550 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph 2024-05-11T01:01:23.551 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-4 2024-05-11T01:01:23.551 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-5 2024-05-11T01:01:23.551 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-6 2024-05-11T01:01:23.551 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-7 2024-05-11T01:01:23.551 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns 2024-05-11T01:01:23.551 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:23.551 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:23.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:23.551 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2024-05-11T01:01:23.557 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:2024-05-11T01:01:23.556+0000 7fa599bc3480 -1 init, newargv = 0x557c4f0fd1c0 newargc=15 2024-05-11T01:01:23.557 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:ceph-fuse[42159]: starting ceph client 2024-05-11T01:01:23.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:ceph-fuse[42159]: starting fuse 2024-05-11T01:01:23.571 INFO:teuthology.orchestra.run.smithi119.stdout:40 2024-05-11T01:01:23.572 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-11T01:01:23.572 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer -- python3 -c ' 2024-05-11T01:01:23.572 DEBUG:teuthology.orchestra.run.smithi119:> import glob 2024-05-11T01:01:23.572 DEBUG:teuthology.orchestra.run.smithi119:> import re 2024-05-11T01:01:23.572 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-05-11T01:01:23.572 DEBUG:teuthology.orchestra.run.smithi119:> import subprocess 2024-05-11T01:01:23.572 DEBUG:teuthology.orchestra.run.smithi119:> 2024-05-11T01:01:23.572 DEBUG:teuthology.orchestra.run.smithi119:> def _find_admin_socket(client_name): 2024-05-11T01:01:23.572 DEBUG:teuthology.orchestra.run.smithi119:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-11T01:01:23.572 DEBUG:teuthology.orchestra.run.smithi119:> files = glob.glob(asok_path) 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> # Given a non-glob path, it better be there 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> if "*" not in asok_path: 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> assert(len(files) == 1) 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> return files[0] 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> for f in files: 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> contents = proc_f.read() 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> if mountpoint in contents: 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> return f 2024-05-11T01:01:23.573 DEBUG:teuthology.orchestra.run.smithi119:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T01:01:23.574 DEBUG:teuthology.orchestra.run.smithi119:> 2024-05-11T01:01:23.574 DEBUG:teuthology.orchestra.run.smithi119:> print(_find_admin_socket("client.1")) 2024-05-11T01:01:23.574 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-05-11T01:01:23.690 INFO:teuthology.orchestra.run.smithi119.stdout:/var/run/ceph/ceph-client.1.42159.asok 2024-05-11T01:01:23.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:23.703 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.42159.asok 2024-05-11T01:01:23.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:23.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.42159.asok status 2024-05-11T01:01:23.839 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "metadata": { 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_sha1": "10380055a1fc5e7d3b37b252af7045f5c2c8ba29", 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.0.0-3570-g10380055 (10380055a1fc5e7d3b37b252af7045f5c2c8ba29) squid (dev)", 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "entity_id": "1", 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": "smithi119", 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "pid": "42159", 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "root": "/" 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_count": 0, 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_pinned_count": 0, 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4481, 2024-05-11T01:01:23.840 INFO:teuthology.orchestra.run.smithi119.stdout: "inst": { 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "name": { 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "client", 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "num": 4481 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3435041482 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2024-05-11T01:01:23.841 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3435041482 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout: "inst_str": "client.4481 192.168.0.1:0/3435041482", 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout: "addr_str": "192.168.0.1:0/3435041482", 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout: "inode_count": 1, 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_epoch": 8, 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch": 20, 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch_barrier": 0, 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklisted": false, 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs" 2024-05-11T01:01:23.842 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-11T01:01:23.848 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T01:01:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.965+0000 7fd1e899e640 1 Processor -- start 2024-05-11T01:01:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.965+0000 7fd1e899e640 1 -- start start 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e899e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e415aa20 0x7fd1e40cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e899e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd1e40cdc80 0x7fd1e4160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e899e640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e40741e0 con 0x7fd1e406df80 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e899e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e415ae00 con 0x7fd1e40cdc80 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e899e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e4073c20 con 0x7fd1e415aa20 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e3fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd1e40cdc80 0x7fd1e4160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e2ffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd1e406df80 0x7fd1e40cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:59604/0 (socket says 172.21.15.52:59604) 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e3fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd1e40cdc80 0x7fd1e4160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43376/0 (socket says 172.21.15.52:43376) 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e2ffd640 1 -- 172.21.15.52:0/3218079551 learned_addr learned my addr 172.21.15.52:0/3218079551 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e37fe640 1 --2- 172.21.15.52:0/3218079551 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e415aa20 0x7fd1e40cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e3fff640 1 -- 172.21.15.52:0/3218079551 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e415aa20 msgr2=0x7fd1e40cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e3fff640 1 --2- 172.21.15.52:0/3218079551 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e415aa20 0x7fd1e40cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e3fff640 1 -- 172.21.15.52:0/3218079551 >> v1:172.21.15.52:6789/0 conn(0x7fd1e406df80 legacy=0x7fd1e40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e3fff640 1 -- 172.21.15.52:0/3218079551 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e4073f00 con 0x7fd1e40cdc80 2024-05-11T01:01:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.966+0000 7fd1e27fc640 1 -- 172.21.15.52:0/3218079551 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 696036278 0 0) 0x7fd1e40741e0 con 0x7fd1e406df80 2024-05-11T01:01:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.967+0000 7fd1e3fff640 1 --2- 172.21.15.52:0/3218079551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 0x7fd1e4160a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009d90 tx=0x7fd1dc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd46d871f771f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.967+0000 7fd1e27fc640 1 -- 172.21.15.52:0/3218079551 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc007740 con 0x7fd1e40cdc80 2024-05-11T01:01:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.967+0000 7fd1e27fc640 1 -- 172.21.15.52:0/3218079551 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0078e0 con 0x7fd1e40cdc80 2024-05-11T01:01:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.967+0000 7fd1e27fc640 1 -- 172.21.15.52:0/3218079551 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc007cb0 con 0x7fd1e40cdc80 2024-05-11T01:01:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.968+0000 7fd1e899e640 1 -- 172.21.15.52:0/3218079551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 msgr2=0x7fd1e4160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.968+0000 7fd1e899e640 1 --2- 172.21.15.52:0/3218079551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 0x7fd1e4160a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009d90 tx=0x7fd1dc02a660 comp rx=0 tx=0).stop 2024-05-11T01:01:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.968+0000 7fd1e899e640 1 -- 172.21.15.52:0/3218079551 shutdown_connections 2024-05-11T01:01:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.968+0000 7fd1e899e640 1 --2- 172.21.15.52:0/3218079551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 0x7fd1e4160a10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.968+0000 7fd1e899e640 1 --2- 172.21.15.52:0/3218079551 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e415aa20 0x7fd1e40cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.968+0000 7fd1e899e640 1 -- 172.21.15.52:0/3218079551 >> 172.21.15.52:0/3218079551 conn(0x7fd1e40b26f0 msgr2=0x7fd1e40b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.968+0000 7fd1e899e640 1 -- 172.21.15.52:0/3218079551 shutdown_connections 2024-05-11T01:01:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.968+0000 7fd1e899e640 1 -- 172.21.15.52:0/3218079551 wait complete. 2024-05-11T01:01:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.969+0000 7fd1e899e640 1 Processor -- start 2024-05-11T01:01:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.969+0000 7fd1e899e640 1 -- start start 2024-05-11T01:01:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e899e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e406df80 0x7fd1e40b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e899e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 0x7fd1e40b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e899e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e415aa20 0x7fd1e40b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e899e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e415d420 con 0x7fd1e40cdc80 2024-05-11T01:01:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e899e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e406fe80 con 0x7fd1e415aa20 2024-05-11T01:01:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e899e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e415d5a0 con 0x7fd1e406df80 2024-05-11T01:01:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 0x7fd1e40b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 0x7fd1e40b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43384/0 (socket says 172.21.15.52:43384) 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e2ffd640 1 -- 172.21.15.52:0/1203798200 learned_addr learned my addr 172.21.15.52:0/1203798200 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e37fe640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e406df80 0x7fd1e40b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.970+0000 7fd1e3fff640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e415aa20 0x7fd1e40b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1e2ffd640 1 -- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e406df80 msgr2=0x7fd1e40b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1e2ffd640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e406df80 0x7fd1e40b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1e2ffd640 1 -- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e415aa20 msgr2=0x7fd1e40b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1e2ffd640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e415aa20 0x7fd1e40b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1e2ffd640 1 -- 172.21.15.52:0/1203798200 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e40b5da0 con 0x7fd1e40cdc80 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1e2ffd640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 0x7fd1e40b37e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800c490 tx=0x7fd1d800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1cbfff640 1 -- 172.21.15.52:0/1203798200 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8018070 con 0x7fd1e40cdc80 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1cbfff640 1 -- 172.21.15.52:0/1203798200 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d8007500 con 0x7fd1e40cdc80 2024-05-11T01:01:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e40b6080 con 0x7fd1e40cdc80 2024-05-11T01:01:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1cbfff640 1 -- 172.21.15.52:0/1203798200 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8004a20 con 0x7fd1e40cdc80 2024-05-11T01:01:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.971+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e40cb1b0 con 0x7fd1e40cdc80 2024-05-11T01:01:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.972+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e40694f0 con 0x7fd1e40cdc80 2024-05-11T01:01:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.973+0000 7fd1cbfff640 1 -- 172.21.15.52:0/1203798200 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd1d801b070 con 0x7fd1e40cdc80 2024-05-11T01:01:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.973+0000 7fd1cbfff640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd1d00762d0 0x7fd1d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.974+0000 7fd1cbfff640 1 -- 172.21.15.52:0/1203798200 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd1d8096920 con 0x7fd1e40cdc80 2024-05-11T01:01:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.974+0000 7fd1e37fe640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd1d00762d0 0x7fd1d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.974+0000 7fd1e37fe640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd1d00762d0 0x7fd1d0078790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40079a0 tx=0x7fd1d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:23.978+0000 7fd1cbfff640 1 -- 172.21.15.52:0/1203798200 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd1d805fb90 con 0x7fd1e40cdc80 2024-05-11T01:01:24.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.207+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd1e40b6250 con 0x7fd1e40cdc80 2024-05-11T01:01:24.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.207+0000 7fd1cbfff640 1 -- 172.21.15.52:0/1203798200 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fd1d8063e90 con 0x7fd1e40cdc80 2024-05-11T01:01:24.207 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T01:01:24.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd1d00762d0 msgr2=0x7fd1d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:24.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd1d00762d0 0x7fd1d0078790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40079a0 tx=0x7fd1d4008040 comp rx=0 tx=0).stop 2024-05-11T01:01:24.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 msgr2=0x7fd1e40b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:24.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 0x7fd1e40b37e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800c490 tx=0x7fd1d800c950 comp rx=0 tx=0).stop 2024-05-11T01:01:24.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 shutdown_connections 2024-05-11T01:01:24.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd1d00762d0 0x7fd1d0078790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:24.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e415aa20 0x7fd1e40b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:24.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e40cdc80 0x7fd1e40b37e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:24.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 --2- 172.21.15.52:0/1203798200 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd1e406df80 0x7fd1e40b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:24.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 >> 172.21.15.52:0/1203798200 conn(0x7fd1e40b26f0 msgr2=0x7fd1e415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:24.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 shutdown_connections 2024-05-11T01:01:24.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:24.209+0000 7fd1e899e640 1 -- 172.21.15.52:0/1203798200 wait complete. 2024-05-11T01:01:24.220 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T01:01:24.220 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T01:01:24.220 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-05-11T01:01:24.220 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-11T01:01:24.221 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-11T01:01:24.221 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T01:01:24.221 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T01:01:24.221 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T01:01:24.221 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-11T01:01:24.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:24.221 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-05-11T01:01:24.238 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:01:24.238 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:01:24.238 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:01:24.238 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.238 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-05-11T01:01:24.238 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-11T01:01:24.239 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42765sec preferred_lft 42765sec 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 2e:ee:72:4b:b9:37 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3814:bdff:fee3:463d/64 scope link 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 2e:ee:72:4b:b9:37 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::2cee:72ff:fe4b:b937/64 scope link tentative 2024-05-11T01:01:24.240 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:24.241 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T01:01:24.302 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-05-11T01:01:24.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:24.304 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-05-11T01:01:24.361 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-05-11T01:01:24.361 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T01:01:24.361 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T01:01:24.362 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T01:01:24.362 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T01:01:24.362 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:24.362 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-11T01:01:24.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T01:01:24.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T01:01:24.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T01:01:24.362 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-11T01:01:24.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:24.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:24.488 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-05-11T01:01:24.504 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T01:01:24.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:24.504 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2024-05-11T01:01:24.559 INFO:teuthology.orchestra.run.smithi119.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-11T01:01:24.560 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:24.560 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-11T01:01:24.560 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T01:01:24.560 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-11T01:01:24.560 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-11T01:01:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:24.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:24.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:24.677 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-11T01:01:24.702 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:01:24.702 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:01:24.702 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:01:24.702 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.702 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-05-11T01:01:24.702 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.703 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T01:01:24.703 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 66:76:63:bb:4d:1d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T01:01:24.703 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T01:01:24.703 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.703 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::6476:63ff:febb:4d1d/64 scope link 2024-05-11T01:01:24.703 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:24.704 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-11T01:01:24.769 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:01:24.769 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:01:24.769 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:01:24.769 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.769 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-05-11T01:01:24.769 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.769 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T01:01:24.769 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 66:76:63:bb:4d:1d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T01:01:24.770 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T01:01:24.770 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.770 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::6476:63ff:febb:4d1d/64 scope link 2024-05-11T01:01:24.770 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:24.771 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-11T01:01:24.771 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:24.771 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-11T01:01:24.771 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-11T01:01:24.771 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-11T01:01:24.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-11T01:01:24.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-11T01:01:24.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-11T01:01:24.772 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-11T01:01:24.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:24.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:24.924 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:24.924 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-11T01:01:24.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.1 up 2024-05-11T01:01:24.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.1 master ceph-brx 2024-05-11T01:01:24.924 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-11T01:01:24.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:25.001 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-11T01:01:25.001 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:25.002 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-11T01:01:25.019 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-11T01:01:25.019 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:25.019 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-11T01:01:25.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2024-05-11T01:01:25.140 DEBUG:teuthology.orchestra.run.smithi119:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:25.196 INFO:teuthology.orchestra.run.smithi119.stdout:/proc 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/sys 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/dev 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/security 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/shm 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/pts 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/run 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/pstore 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/bpf 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/ 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/selinux 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/hugepages 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/mqueue 2024-05-11T01:01:25.197 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/debug 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/tracing 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/config 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/1000 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-4 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-5 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-6 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-7 2024-05-11T01:01:25.198 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns 2024-05-11T01:01:25.199 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:25.199 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:25.199 INFO:teuthology.orchestra.run.smithi119.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:25.199 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T01:01:25.199 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T01:01:25.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.199 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2024-05-11T01:01:25.252 INFO:teuthology.orchestra.run.smithi119.stdout:40 2024-05-11T01:01:25.252 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-05-11T01:01:25.252 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-11T01:01:25.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2024-05-11T01:01:25.320 DEBUG:teuthology.orchestra.run.smithi119:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:25.377 INFO:teuthology.orchestra.run.smithi119.stdout:/proc 2024-05-11T01:01:25.377 INFO:teuthology.orchestra.run.smithi119.stdout:/sys 2024-05-11T01:01:25.377 INFO:teuthology.orchestra.run.smithi119.stdout:/dev 2024-05-11T01:01:25.377 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/security 2024-05-11T01:01:25.377 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/shm 2024-05-11T01:01:25.377 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/pts 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/run 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/pstore 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/bpf 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/ 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/selinux 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/hugepages 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/mqueue 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/debug 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/tracing 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:25.378 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/config 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/1000 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-4 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-5 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-6 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-7 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T01:01:25.379 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T01:01:25.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi119.stderr:2024-05-11T01:01:25.377+0000 7f4fac673480 -1 init, newargv = 0x55f51cdc4380 newargc=15 2024-05-11T01:01:25.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.381 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2024-05-11T01:01:25.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi119.stderr:ceph-fuse[42615]: starting ceph client 2024-05-11T01:01:25.382 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi119.stderr:ceph-fuse[42615]: starting fuse 2024-05-11T01:01:25.396 INFO:teuthology.orchestra.run.smithi119.stdout:40 2024-05-11T01:01:25.397 INFO:teuthology.orchestra.run.smithi119.stdout:43 2024-05-11T01:01:25.397 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 43] 2024-05-11T01:01:25.397 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer -- python3 -c ' 2024-05-11T01:01:25.397 DEBUG:teuthology.orchestra.run.smithi119:> import glob 2024-05-11T01:01:25.397 DEBUG:teuthology.orchestra.run.smithi119:> import re 2024-05-11T01:01:25.397 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-05-11T01:01:25.397 DEBUG:teuthology.orchestra.run.smithi119:> import subprocess 2024-05-11T01:01:25.397 DEBUG:teuthology.orchestra.run.smithi119:> 2024-05-11T01:01:25.397 DEBUG:teuthology.orchestra.run.smithi119:> def _find_admin_socket(client_name): 2024-05-11T01:01:25.397 DEBUG:teuthology.orchestra.run.smithi119:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-11T01:01:25.397 DEBUG:teuthology.orchestra.run.smithi119:> files = glob.glob(asok_path) 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> # Given a non-glob path, it better be there 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> if "*" not in asok_path: 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> assert(len(files) == 1) 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> return files[0] 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> for f in files: 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> contents = proc_f.read() 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> if mountpoint in contents: 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> return f 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T01:01:25.398 DEBUG:teuthology.orchestra.run.smithi119:> 2024-05-11T01:01:25.399 DEBUG:teuthology.orchestra.run.smithi119:> print(_find_admin_socket("client.3")) 2024-05-11T01:01:25.399 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-05-11T01:01:25.515 INFO:teuthology.orchestra.run.smithi119.stdout:/var/run/ceph/ceph-client.3.42615.asok 2024-05-11T01:01:25.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-11T01:01:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:25.526 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.42615.asok 2024-05-11T01:01:25.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.527 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.42615.asok status 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "metadata": { 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_sha1": "10380055a1fc5e7d3b37b252af7045f5c2c8ba29", 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.0.0-3570-g10380055 (10380055a1fc5e7d3b37b252af7045f5c2c8ba29) squid (dev)", 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "entity_id": "3", 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": "smithi119", 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "pid": "42615", 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "root": "/" 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_count": 0, 2024-05-11T01:01:25.661 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_pinned_count": 0, 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4621, 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "inst": { 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "name": { 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "client", 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "num": 4621 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.2:0", 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3023434223 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2024-05-11T01:01:25.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.2:0", 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3023434223 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "inst_str": "client.4621 192.168.0.2:0/3023434223", 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "addr_str": "192.168.0.2:0/3023434223", 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "inode_count": 1, 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_epoch": 8, 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch": 20, 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch_barrier": 0, 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklisted": false, 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs" 2024-05-11T01:01:25.663 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-11T01:01:25.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.669 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-05-11T01:01:25.669 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:25.690 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-05-11T01:01:25.691 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:25.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.691 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:25.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.763 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-05-11T01:01:25.763 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-11T01:01:25.822 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-05-11T01:01:25.823 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-11T01:01:25.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.823 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-11T01:01:25.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.892 DEBUG:teuthology.orchestra.run.smithi119:> stat --file-system '--printf=%T 2024-05-11T01:01:25.892 DEBUG:teuthology.orchestra.run.smithi119:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:25.908 INFO:teuthology.orchestra.run.smithi119.stdout:fuseblk 2024-05-11T01:01:25.908 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:25.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:25.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:25.974 DEBUG:teuthology.orchestra.run.smithi119:> stat --file-system '--printf=%T 2024-05-11T01:01:25.974 DEBUG:teuthology.orchestra.run.smithi119:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-11T01:01:26.031 INFO:teuthology.orchestra.run.smithi119.stdout:fuseblk 2024-05-11T01:01:26.031 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-11T01:01:26.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:26.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-11T01:01:26.097 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-11T01:01:26.243 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2024-05-11T01:01:26.243 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-05-11T01:01:26.244 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:26.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.361+0000 7fa0204da640 1 Processor -- start 2024-05-11T01:01:26.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.361+0000 7fa0204da640 1 -- start start 2024-05-11T01:01:26.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa0204da640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa018160810 0x7fa0180cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa0204da640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0180ce3c0 con 0x7fa0180d3280 2024-05-11T01:01:26.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa0204da640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0180d0dc0 con 0x7fa0180cdf10 2024-05-11T01:01:26.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa0204da640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa0180ce640 con 0x7fa018160810 2024-05-11T01:01:26.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa01ea77640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa0180d3280 0x7fa0180b2f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:59612/0 (socket says 172.21.15.52:59612) 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa01ea77640 1 -- 172.21.15.52:0/3103281214 learned_addr learned my addr 172.21.15.52:0/3103281214 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa01e276640 1 --2- 172.21.15.52:0/3103281214 >> v2:172.21.15.119:3300/0 conn(0x7fa018160810 0x7fa0180cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2589144956 0 0) 0x7fa010001490 con 0x7fa0180d3280 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa004003520 con 0x7fa0180d3280 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.362+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 121128891 0 0) 0x7fa004003520 con 0x7fa0180d3280 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 >> v2:172.21.15.119:3300/0 conn(0x7fa018160810 msgr2=0x7fa0180cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 --2- 172.21.15.52:0/3103281214 >> v2:172.21.15.119:3300/0 conn(0x7fa018160810 0x7fa0180cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 >> v1:172.21.15.119:6789/0 conn(0x7fa0180cdf10 legacy=0x7fa0180d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0180ce920 con 0x7fa0180d3280 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fa010001c40 con 0x7fa0180d3280 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 >> v1:172.21.15.52:6789/0 conn(0x7fa0180d3280 legacy=0x7fa0180b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa004003a50 0x7fa004003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0040051c0 0x7fa00400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0180ce640 con 0x7fa004004800 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa00400c160 con 0x7fa0040051c0 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa00400c360 con 0x7fa004003a50 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01da75640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa004004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01e276640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa004003a50 0x7fa004003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01ea77640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0040051c0 0x7fa00400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01da75640 1 -- 172.21.15.52:0/3103281214 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa004003a50 msgr2=0x7fa004003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01da75640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa004003a50 0x7fa004003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01da75640 1 -- 172.21.15.52:0/3103281214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0040051c0 msgr2=0x7fa00400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01da75640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0040051c0 0x7fa00400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01ea77640 1 -- 172.21.15.52:0/3103281214 reap_dead start 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.363+0000 7fa01da75640 1 -- 172.21.15.52:0/3103281214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa004003520 con 0x7fa004004800 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.364+0000 7fa01da75640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa004004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa008001790 tx=0x7fa004005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc099b1a8528d75a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.364+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0080013b0 con 0x7fa004004800 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.364+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0080052d0 con 0x7fa004004800 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.364+0000 7fa01d274640 1 -- 172.21.15.52:0/3103281214 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00802aac0 con 0x7fa004004800 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.364+0000 7fa0204da640 1 -- 172.21.15.52:0/3103281214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 msgr2=0x7fa004004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.364+0000 7fa0204da640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa004004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa008001790 tx=0x7fa004005830 comp rx=0 tx=0).stop 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.365+0000 7fa0204da640 1 -- 172.21.15.52:0/3103281214 shutdown_connections 2024-05-11T01:01:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.365+0000 7fa0204da640 1 --2- 172.21.15.52:0/3103281214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa004004c80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.365+0000 7fa0204da640 1 -- 172.21.15.52:0/3103281214 >> 172.21.15.52:0/3103281214 conn(0x7fa01809ba20 msgr2=0x7fa01809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:26.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.365+0000 7fa0204da640 1 -- 172.21.15.52:0/3103281214 shutdown_connections 2024-05-11T01:01:26.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.365+0000 7fa0204da640 1 -- 172.21.15.52:0/3103281214 wait complete. 2024-05-11T01:01:26.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.366+0000 7fa0204da640 1 Processor -- start 2024-05-11T01:01:26.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.366+0000 7fa0204da640 1 -- start start 2024-05-11T01:01:26.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa0204da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa0180b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa0204da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa018154d90 0x7fa0180ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa0204da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0180c9090 0x7fa0180c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa0204da640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0180cd9d0 con 0x7fa004004800 2024-05-11T01:01:26.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa0204da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0180b2a10 con 0x7fa0180c9090 2024-05-11T01:01:26.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa0204da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa0180cdb50 con 0x7fa018154d90 2024-05-11T01:01:26.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01e276640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa0180b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01e276640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa0180b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43406/0 (socket says 172.21.15.52:43406) 2024-05-11T01:01:26.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01e276640 1 -- 172.21.15.52:0/2742393756 learned_addr learned my addr 172.21.15.52:0/2742393756 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:26.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01ea77640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0180c9090 0x7fa0180c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01da75640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa018154d90 0x7fa0180ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01e276640 1 -- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa018154d90 msgr2=0x7fa0180ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01e276640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa018154d90 0x7fa0180ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01e276640 1 -- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0180c9090 msgr2=0x7fa0180c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01e276640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0180c9090 0x7fa0180c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01e276640 1 -- 172.21.15.52:0/2742393756 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0180b8550 con 0x7fa004004800 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01ea77640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0180c9090 0x7fa0180c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01da75640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa018154d90 0x7fa0180ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa01e276640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa0180b8010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa014011890 tx=0x7fa014011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.367+0000 7fa00effd640 1 -- 172.21.15.52:0/2742393756 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014019070 con 0x7fa004004800 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.368+0000 7fa00effd640 1 -- 172.21.15.52:0/2742393756 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa014004240 con 0x7fa004004800 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.368+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0180cab30 con 0x7fa004004800 2024-05-11T01:01:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.368+0000 7fa00effd640 1 -- 172.21.15.52:0/2742393756 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014009590 con 0x7fa004004800 2024-05-11T01:01:26.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.368+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0180c5c10 con 0x7fa004004800 2024-05-11T01:01:26.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.368+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa018069780 con 0x7fa004004800 2024-05-11T01:01:26.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.369+0000 7fa00effd640 1 -- 172.21.15.52:0/2742393756 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa01401c070 con 0x7fa004004800 2024-05-11T01:01:26.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.370+0000 7fa00effd640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9ffc0762d0 0x7f9ffc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.370+0000 7fa00effd640 1 -- 172.21.15.52:0/2742393756 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa014097930 con 0x7fa004004800 2024-05-11T01:01:26.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.370+0000 7fa01da75640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9ffc0762d0 0x7f9ffc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.370+0000 7fa01da75640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9ffc0762d0 0x7f9ffc078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa008032000 tx=0x7fa00802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:26.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.374+0000 7fa00effd640 1 -- 172.21.15.52:0/2742393756 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa0180cab30 con 0x7fa004004800 2024-05-11T01:01:26.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.567+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0180b1c10 con 0x7fa004004800 2024-05-11T01:01:26.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.568+0000 7fa00effd640 1 -- 172.21.15.52:0/2742393756 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fa014061320 con 0x7fa004004800 2024-05-11T01:01:26.568 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:26.569 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":20,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:02.673511+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:00:59.086890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:00.008780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:26.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9ffc0762d0 msgr2=0x7f9ffc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9ffc0762d0 0x7f9ffc078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa008032000 tx=0x7fa00802e000 comp rx=0 tx=0).stop 2024-05-11T01:01:26.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 msgr2=0x7fa0180b8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa0180b8010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa014011890 tx=0x7fa014011d50 comp rx=0 tx=0).stop 2024-05-11T01:01:26.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 shutdown_connections 2024-05-11T01:01:26.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9ffc0762d0 0x7f9ffc078790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0180c9090 0x7fa0180c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa018154d90 0x7fa0180ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 --2- 172.21.15.52:0/2742393756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa004004800 0x7fa0180b8010 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 >> 172.21.15.52:0/2742393756 conn(0x7fa01809ba20 msgr2=0x7fa0180d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:26.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 shutdown_connections 2024-05-11T01:01:26.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.570+0000 7fa0204da640 1 -- 172.21.15.52:0/2742393756 wait complete. 2024-05-11T01:01:26.582 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T01:00:55.525012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-11T01:00:59.086890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-11T01:01:00.008780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T01:01:26.583 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T01:01:26.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.690+0000 7ffa42cc8640 1 Processor -- start 2024-05-11T01:01:26.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.690+0000 7ffa42cc8640 1 -- start start 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa42cc8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa3c072330 0x7ffa3c0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa42cc8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa3c06c480 0x7ffa3c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa42cc8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa3c06f700 con 0x7ffa3c15d0e0 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa42cc8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffa3c06f9e0 con 0x7ffa3c06c480 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa42cc8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3c06ff80 con 0x7ffa3c072330 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa40a64640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7ffa3c15d0e0 0x7ffa3c0ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:59624/0 (socket says 172.21.15.52:59624) 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa40a64640 1 -- 172.21.15.52:0/1363278305 learned_addr learned my addr 172.21.15.52:0/1363278305 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa41265640 1 --2- 172.21.15.52:0/1363278305 >> v2:172.21.15.119:3300/0 conn(0x7ffa3c06c480 0x7ffa3c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa33fff640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa3c072330 0x7ffa3c0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3362417465 0 0) 0x7ffa3c06f700 con 0x7ffa3c15d0e0 2024-05-11T01:01:26.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa24003520 con 0x7ffa3c15d0e0 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1793540834 0 0) 0x7ffa24003520 con 0x7ffa3c15d0e0 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa3c072330 msgr2=0x7ffa3c0b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa3c072330 0x7ffa3c0b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 >> v2:172.21.15.119:3300/0 conn(0x7ffa3c06c480 msgr2=0x7ffa3c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 --2- 172.21.15.52:0/1363278305 >> v2:172.21.15.119:3300/0 conn(0x7ffa3c06c480 0x7ffa3c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c070260 con 0x7ffa3c15d0e0 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7ffa38001a70 con 0x7ffa3c15d0e0 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 >> v1:172.21.15.52:6789/0 conn(0x7ffa3c15d0e0 legacy=0x7ffa3c0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa24003a00 0x7ffa24003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.691+0000 7ffa337fe640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa24004800 0x7ffa24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa337fe640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa3c072330 0x7ffa240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c06f9e0 con 0x7ffa24004800 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c06ff80 con 0x7ffa24003a00 2024-05-11T01:01:26.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2400b9c0 con 0x7ffa3c072330 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa33fff640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa24004800 0x7ffa24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa40a64640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa24003a00 0x7ffa24003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa41265640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa3c072330 0x7ffa240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa33fff640 1 -- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa3c072330 msgr2=0x7ffa240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa33fff640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa3c072330 0x7ffa240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa33fff640 1 -- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa24003a00 msgr2=0x7ffa24003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa33fff640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa24003a00 0x7ffa24003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.692+0000 7ffa33fff640 1 -- 172.21.15.52:0/1363278305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa24003520 con 0x7ffa24004800 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa33fff640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa24004800 0x7ffa24004c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009af0 tx=0x7ffa24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2dc2115b8c6d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c036070 con 0x7ffa24004800 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa2c031400 con 0x7ffa24004800 2024-05-11T01:01:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa337fe640 1 -- 172.21.15.52:0/1363278305 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c009690 con 0x7ffa24004800 2024-05-11T01:01:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa42cc8640 1 -- 172.21.15.52:0/1363278305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa24004800 msgr2=0x7ffa24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa42cc8640 1 --2- 172.21.15.52:0/1363278305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa24004800 0x7ffa24004c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009af0 tx=0x7ffa24005830 comp rx=0 tx=0).stop 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa41265640 1 -- 172.21.15.52:0/1363278305 reap_dead start 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa42cc8640 1 -- 172.21.15.52:0/1363278305 shutdown_connections 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa42cc8640 1 -- 172.21.15.52:0/1363278305 >> 172.21.15.52:0/1363278305 conn(0x7ffa3c0cd830 msgr2=0x7ffa3c0cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa42cc8640 1 -- 172.21.15.52:0/1363278305 shutdown_connections 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.693+0000 7ffa42cc8640 1 -- 172.21.15.52:0/1363278305 wait complete. 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.695+0000 7ffa42cc8640 1 Processor -- start 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.695+0000 7ffa42cc8640 1 -- start start 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa42cc8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa24004800 0x7ffa3c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa42cc8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa3c0c78f0 0x7ffa3c09e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa42cc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3c09e630 0x7ffa3c0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa42cc8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c0b0380 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa42cc8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c0b0200 con 0x7ffa3c09e630 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa42cc8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3c0b0500 con 0x7ffa24004800 2024-05-11T01:01:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa33fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa3c0c78f0 0x7ffa3c09e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa33fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa3c0c78f0 0x7ffa3c09e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43418/0 (socket says 172.21.15.52:43418) 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa33fff640 1 -- 172.21.15.52:0/3823579456 learned_addr learned my addr 172.21.15.52:0/3823579456 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa40a64640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa24004800 0x7ffa3c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa41265640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3c09e630 0x7ffa3c0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa33fff640 1 -- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa24004800 msgr2=0x7ffa3c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa33fff640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa24004800 0x7ffa3c15b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa33fff640 1 -- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3c09e630 msgr2=0x7ffa3c0b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa33fff640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3c09e630 0x7ffa3c0b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa33fff640 1 -- 172.21.15.52:0/3823579456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c09ead0 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa33fff640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa3c0c78f0 0x7ffa3c09e0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c034040 tx=0x7ffa2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa317fa640 1 -- 172.21.15.52:0/3823579456 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c041020 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa317fa640 1 -- 172.21.15.52:0/3823579456 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa2c03c470 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c0c4af0 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa317fa640 1 -- 172.21.15.52:0/3823579456 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c036040 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.696+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c15b590 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.697+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3c0c48e0 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.698+0000 7ffa317fa640 1 -- 172.21.15.52:0/3823579456 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffa2c044020 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.698+0000 7ffa317fa640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa280762d0 0x7ffa28078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.699+0000 7ffa317fa640 1 -- 172.21.15.52:0/3823579456 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa2c0b7b90 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.699+0000 7ffa40a64640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa280762d0 0x7ffa28078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.699+0000 7ffa40a64640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa280762d0 0x7ffa28078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffa380012e0 tx=0x7ffa38006040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.703+0000 7ffa317fa640 1 -- 172.21.15.52:0/3823579456 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffa2c04b030 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.894+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa3c0d02e0 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.894+0000 7ffa317fa640 1 -- 172.21.15.52:0/3823579456 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ffa2c0815b0 con 0x7ffa3c0c78f0 2024-05-11T01:01:26.895 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa280762d0 msgr2=0x7ffa28078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa280762d0 0x7ffa28078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffa380012e0 tx=0x7ffa38006040 comp rx=0 tx=0).stop 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa3c0c78f0 msgr2=0x7ffa3c09e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa3c0c78f0 0x7ffa3c09e0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c034040 tx=0x7ffa2c004730 comp rx=0 tx=0).stop 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 shutdown_connections 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffa280762d0 0x7ffa28078790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa3c09e630 0x7ffa3c0b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa3c0c78f0 0x7ffa3c09e0f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 --2- 172.21.15.52:0/3823579456 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa24004800 0x7ffa3c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 >> 172.21.15.52:0/3823579456 conn(0x7ffa3c0cd830 msgr2=0x7ffa3c0d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 shutdown_connections 2024-05-11T01:01:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:26.896+0000 7ffa42cc8640 1 -- 172.21.15.52:0/3823579456 wait complete. 2024-05-11T01:01:26.908 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-11T01:01:27.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.014+0000 7fb1d80e8640 1 Processor -- start 2024-05-11T01:01:27.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.015+0000 7fb1d80e8640 1 -- start start 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.015+0000 7fb1d80e8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d00cd7f0 0x7fb1d0073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.015+0000 7fb1d80e8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb1d0073790 0x7fb1d006f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.015+0000 7fb1d80e8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d00cde60 con 0x7fb1d015c430 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.015+0000 7fb1d80e8640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb1d00ce140 con 0x7fb1d0073790 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.015+0000 7fb1d80e8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d00ce420 con 0x7fb1d00cd7f0 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.015+0000 7fb1d6685640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb1d0073790 0x7fb1d006f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.015+0000 7fb1d6685640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb1d0073790 0x7fb1d006f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43422/0 (socket says 172.21.15.52:43422) 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.015+0000 7fb1d6685640 1 -- 172.21.15.52:0/2423881891 learned_addr learned my addr 172.21.15.52:0/2423881891 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.016+0000 7fb1d5683640 1 --2- 172.21.15.52:0/2423881891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d00cd7f0 0x7fb1d0073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.016+0000 7fb1d6685640 1 -- 172.21.15.52:0/2423881891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d00cd7f0 msgr2=0x7fb1d0073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.016+0000 7fb1d6685640 1 --2- 172.21.15.52:0/2423881891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d00cd7f0 0x7fb1d0073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.016+0000 7fb1d6685640 1 -- 172.21.15.52:0/2423881891 >> v1:172.21.15.119:6789/0 conn(0x7fb1d015c430 legacy=0x7fb1d00d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.016+0000 7fb1d6685640 1 -- 172.21.15.52:0/2423881891 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d006f170 con 0x7fb1d0073790 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.016+0000 7fb1d5683640 1 --2- 172.21.15.52:0/2423881891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d00cd7f0 0x7fb1d0073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.016+0000 7fb1d6685640 1 --2- 172.21.15.52:0/2423881891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0073790 0x7fb1d006f690 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009e50 tx=0x7fb1c002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58648fd102aae6ae server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d4e82640 1 -- 172.21.15.52:0/2423881891 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0036070 con 0x7fb1d0073790 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d4e82640 1 -- 172.21.15.52:0/2423881891 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0031600 con 0x7fb1d0073790 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d4e82640 1 -- 172.21.15.52:0/2423881891 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00319d0 con 0x7fb1d0073790 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d80e8640 1 -- 172.21.15.52:0/2423881891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0073790 msgr2=0x7fb1d006f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d80e8640 1 --2- 172.21.15.52:0/2423881891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0073790 0x7fb1d006f690 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009e50 tx=0x7fb1c002a680 comp rx=0 tx=0).stop 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d80e8640 1 -- 172.21.15.52:0/2423881891 shutdown_connections 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d80e8640 1 --2- 172.21.15.52:0/2423881891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0073790 0x7fb1d006f690 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d80e8640 1 --2- 172.21.15.52:0/2423881891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d00cd7f0 0x7fb1d0073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d80e8640 1 -- 172.21.15.52:0/2423881891 >> 172.21.15.52:0/2423881891 conn(0x7fb1d00d1ef0 msgr2=0x7fb1d00d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:27.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d80e8640 1 -- 172.21.15.52:0/2423881891 shutdown_connections 2024-05-11T01:01:27.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.017+0000 7fb1d80e8640 1 -- 172.21.15.52:0/2423881891 wait complete. 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.018+0000 7fb1d80e8640 1 Processor -- start 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.018+0000 7fb1d80e8640 1 -- start start 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d80e8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d0073790 0x7fb1d01e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d80e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1d00cd7f0 0x7fb1d00b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d80e8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d015c430 0x7fb1d00b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d80e8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d006ff60 con 0x7fb1d015c430 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d80e8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d006fde0 con 0x7fb1d00cd7f0 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d80e8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d00700e0 con 0x7fb1d0073790 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d6685640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d015c430 0x7fb1d00b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d6685640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d015c430 0x7fb1d00b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43424/0 (socket says 172.21.15.52:43424) 2024-05-11T01:01:27.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d6685640 1 -- 172.21.15.52:0/4032290494 learned_addr learned my addr 172.21.15.52:0/4032290494 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.019+0000 7fb1d5e84640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d0073790 0x7fb1d01e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1d5683640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1d00cd7f0 0x7fb1d00b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1d6685640 1 -- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d0073790 msgr2=0x7fb1d01e8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1d6685640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d0073790 0x7fb1d01e8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1d6685640 1 -- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1d00cd7f0 msgr2=0x7fb1d00b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1d6685640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1d00cd7f0 0x7fb1d00b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1d6685640 1 -- 172.21.15.52:0/4032290494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d00b7a00 con 0x7fb1d015c430 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1d6685640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d015c430 0x7fb1d00b71c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009e50 tx=0x7fb1c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1ce7fc640 1 -- 172.21.15.52:0/4032290494 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0041020 con 0x7fb1d015c430 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1ce7fc640 1 -- 172.21.15.52:0/4032290494 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c00077a0 con 0x7fb1d015c430 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d00aefb0 con 0x7fb1d015c430 2024-05-11T01:01:27.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1ce7fc640 1 -- 172.21.15.52:0/4032290494 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0036040 con 0x7fb1d015c430 2024-05-11T01:01:27.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.020+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d00af570 con 0x7fb1d015c430 2024-05-11T01:01:27.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.021+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d00b7760 con 0x7fb1d015c430 2024-05-11T01:01:27.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.022+0000 7fb1ce7fc640 1 -- 172.21.15.52:0/4032290494 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb1c0044020 con 0x7fb1d015c430 2024-05-11T01:01:27.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.022+0000 7fb1ce7fc640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb1bc0762d0 0x7fb1bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.022+0000 7fb1ce7fc640 1 -- 172.21.15.52:0/4032290494 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb1c00b8400 con 0x7fb1d015c430 2024-05-11T01:01:27.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.023+0000 7fb1d5e84640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb1bc0762d0 0x7fb1bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.023+0000 7fb1d5e84640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb1bc0762d0 0x7fb1bc078790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8006fd0 tx=0x7fb1c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:27.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.027+0000 7fb1ce7fc640 1 -- 172.21.15.52:0/4032290494 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb1c004b030 con 0x7fb1d015c430 2024-05-11T01:01:27.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.215+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb1d0069c00 con 0x7fb1d015c430 2024-05-11T01:01:27.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.216+0000 7fb1ce7fc640 1 -- 172.21.15.52:0/4032290494 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb1c0081cd0 con 0x7fb1d015c430 2024-05-11T01:01:27.216 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-05-11T01:01:27.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb1bc0762d0 msgr2=0x7fb1bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb1bc0762d0 0x7fb1bc078790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8006fd0 tx=0x7fb1c8008040 comp rx=0 tx=0).stop 2024-05-11T01:01:27.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d015c430 msgr2=0x7fb1d00b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d015c430 0x7fb1d00b71c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009e50 tx=0x7fb1c0004a40 comp rx=0 tx=0).stop 2024-05-11T01:01:27.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 shutdown_connections 2024-05-11T01:01:27.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb1bc0762d0 0x7fb1bc078790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d015c430 0x7fb1d00b71c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1d00cd7f0 0x7fb1d00b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 --2- 172.21.15.52:0/4032290494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb1d0073790 0x7fb1d01e8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 >> 172.21.15.52:0/4032290494 conn(0x7fb1d00d1ef0 msgr2=0x7fb1d00cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:27.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 shutdown_connections 2024-05-11T01:01:27.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.218+0000 7fb1d80e8640 1 -- 172.21.15.52:0/4032290494 wait complete. 2024-05-11T01:01:27.229 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-11T01:01:27.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.335+0000 7f4fd0589640 1 Processor -- start 2024-05-11T01:01:27.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.335+0000 7f4fd0589640 1 -- start start 2024-05-11T01:01:27.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fd0589640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4fc80d1970 0x7f4fc80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fd0589640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fc80d0700 0x7f4fc80d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fd0589640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fc80cfc60 con 0x7f4fc815f430 2024-05-11T01:01:27.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fd0589640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f4fc80d2a80 con 0x7f4fc80d1970 2024-05-11T01:01:27.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fd0589640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc8163120 con 0x7f4fc80d0700 2024-05-11T01:01:27.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fce325640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f4fc815f430 0x7f4fc80b4ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:59638/0 (socket says 172.21.15.52:59638) 2024-05-11T01:01:27.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fce325640 1 -- 172.21.15.52:0/3700271323 learned_addr learned my addr 172.21.15.52:0/3700271323 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fcdb24640 1 --2- 172.21.15.52:0/3700271323 >> v2:172.21.15.119:3300/0 conn(0x7f4fc80d1970 0x7f4fc80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fceb26640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fc80d0700 0x7f4fc80d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4189217185 0 0) 0x7f4fc80cfc60 con 0x7f4fc815f430 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.336+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fb4003520 con 0x7f4fc815f430 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 254926971 0 0) 0x7f4fb4003520 con 0x7f4fc815f430 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fc80d0700 msgr2=0x7f4fc80d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fc80d0700 0x7f4fc80d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 >> v2:172.21.15.119:3300/0 conn(0x7f4fc80d1970 msgr2=0x7f4fc80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 --2- 172.21.15.52:0/3700271323 >> v2:172.21.15.119:3300/0 conn(0x7f4fc80d1970 0x7f4fc80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc80d24c0 con 0x7f4fc815f430 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f4fc40016f0 con 0x7f4fc815f430 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 >> v1:172.21.15.52:6789/0 conn(0x7f4fc815f430 legacy=0x7f4fc80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fc80d0700 0x7f4fb4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb4003fd0 0x7f4fb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fb4004d40 0x7f4fb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80d2a80 con 0x7f4fb4004d40 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8163120 con 0x7f4fb4003fd0 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb400ba10 con 0x7f4fc80d0700 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fceb26640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fb4004d40 0x7f4fb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fce325640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fc80d0700 0x7f4fb4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fcdb24640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb4003fd0 0x7f4fb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fceb26640 1 -- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fc80d0700 msgr2=0x7f4fb4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fceb26640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fc80d0700 0x7f4fb4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fceb26640 1 -- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb4003fd0 msgr2=0x7f4fb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fceb26640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb4003fd0 0x7f4fb4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.337+0000 7f4fceb26640 1 -- 172.21.15.52:0/3700271323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb4003520 con 0x7f4fb4004d40 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.338+0000 7f4fceb26640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fb4004d40 0x7f4fb40051c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000e170 tx=0x7f4fb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a3b6b795d410db server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.338+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0034020 con 0x7f4fb4004d40 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.338+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc00305f0 con 0x7f4fb4004d40 2024-05-11T01:01:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.338+0000 7f4fcd323640 1 -- 172.21.15.52:0/3700271323 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0009c10 con 0x7f4fb4004d40 2024-05-11T01:01:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.339+0000 7f4fd0589640 1 -- 172.21.15.52:0/3700271323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fb4004d40 msgr2=0x7f4fb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.339+0000 7f4fd0589640 1 --2- 172.21.15.52:0/3700271323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fb4004d40 0x7f4fb40051c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000e170 tx=0x7f4fb4005830 comp rx=0 tx=0).stop 2024-05-11T01:01:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.339+0000 7f4fceb26640 1 -- 172.21.15.52:0/3700271323 reap_dead start 2024-05-11T01:01:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.339+0000 7f4fd0589640 1 -- 172.21.15.52:0/3700271323 shutdown_connections 2024-05-11T01:01:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.339+0000 7f4fd0589640 1 -- 172.21.15.52:0/3700271323 >> 172.21.15.52:0/3700271323 conn(0x7f4fc808ebb0 msgr2=0x7f4fc809e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.339+0000 7f4fd0589640 1 -- 172.21.15.52:0/3700271323 shutdown_connections 2024-05-11T01:01:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.339+0000 7f4fd0589640 1 -- 172.21.15.52:0/3700271323 wait complete. 2024-05-11T01:01:27.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.340+0000 7f4fd0589640 1 Processor -- start 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.340+0000 7f4fd0589640 1 -- start start 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fd0589640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fb4004d40 0x7f4fc80caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fd0589640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc815f810 0x7f4fc8162220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fd0589640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc80cc260 0x7f4fc80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fd0589640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80d01e0 con 0x7f4fc815f810 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fd0589640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80cff00 con 0x7f4fc80cc260 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fd0589640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc80b55f0 con 0x7f4fb4004d40 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fcdb24640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc815f810 0x7f4fc8162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fcdb24640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc815f810 0x7f4fc8162220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43446/0 (socket says 172.21.15.52:43446) 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fcdb24640 1 -- 172.21.15.52:0/4059553776 learned_addr learned my addr 172.21.15.52:0/4059553776 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fce325640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fb4004d40 0x7f4fc80caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fceb26640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc80cc260 0x7f4fc80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fcdb24640 1 -- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fb4004d40 msgr2=0x7f4fc80caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fcdb24640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fb4004d40 0x7f4fc80caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fcdb24640 1 -- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc80cc260 msgr2=0x7f4fc80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fcdb24640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc80cc260 0x7f4fc80ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fcdb24640 1 -- 172.21.15.52:0/4059553776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc80cb410 con 0x7f4fc815f810 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fcdb24640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc815f810 0x7f4fc8162220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8011960 tx=0x7f4fb8011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fbeffd640 1 -- 172.21.15.52:0/4059553776 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800faf0 con 0x7f4fc815f810 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fbeffd640 1 -- 172.21.15.52:0/4059553776 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb800fc90 con 0x7f4fc815f810 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc8162760 con 0x7f4fc815f810 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.341+0000 7f4fbeffd640 1 -- 172.21.15.52:0/4059553776 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8014440 con 0x7f4fc815f810 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.342+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc80cb990 con 0x7f4fc815f810 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.343+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc80695d0 con 0x7f4fc815f810 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.343+0000 7f4fbeffd640 1 -- 172.21.15.52:0/4059553776 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4fb8012040 con 0x7f4fc815f810 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.344+0000 7f4fbeffd640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4fac0762d0 0x7f4fac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.344+0000 7f4fbeffd640 1 -- 172.21.15.52:0/4059553776 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4fb8097ba0 con 0x7f4fc815f810 2024-05-11T01:01:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.344+0000 7f4fce325640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4fac0762d0 0x7f4fac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.344+0000 7f4fce325640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4fac0762d0 0x7f4fac078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4001010 tx=0x7f4fc4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.348+0000 7f4fbeffd640 1 -- 172.21.15.52:0/4059553776 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4fb8019030 con 0x7f4fc815f810 2024-05-11T01:01:27.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.540+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4fc80b6440 con 0x7f4fc815f810 2024-05-11T01:01:27.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.540+0000 7f4fbeffd640 1 -- 172.21.15.52:0/4059553776 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4fb8061590 con 0x7f4fc815f810 2024-05-11T01:01:27.540 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-05-11T01:01:27.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4fac0762d0 msgr2=0x7f4fac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4fac0762d0 0x7f4fac078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4001010 tx=0x7f4fc4009040 comp rx=0 tx=0).stop 2024-05-11T01:01:27.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc815f810 msgr2=0x7f4fc8162220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc815f810 0x7f4fc8162220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8011960 tx=0x7f4fb8011e20 comp rx=0 tx=0).stop 2024-05-11T01:01:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 shutdown_connections 2024-05-11T01:01:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f4fac0762d0 0x7f4fac078790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc80cc260 0x7f4fc80ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc815f810 0x7f4fc8162220 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 --2- 172.21.15.52:0/4059553776 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4fb4004d40 0x7f4fc80caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 >> 172.21.15.52:0/4059553776 conn(0x7f4fc808ebb0 msgr2=0x7f4fc806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 shutdown_connections 2024-05-11T01:01:27.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.542+0000 7f4fd0589640 1 -- 172.21.15.52:0/4059553776 wait complete. 2024-05-11T01:01:27.553 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2024-05-11T01:01:27.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.657+0000 7f385e932640 1 Processor -- start 2024-05-11T01:01:27.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.657+0000 7f385e932640 1 -- start start 2024-05-11T01:01:27.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.658+0000 7f385e932640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f38580d0e90 0x7f38580717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.658+0000 7f385e932640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3858073250 0x7f38580736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.658+0000 7f385e932640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385815fe50 con 0x7f385815c3f0 2024-05-11T01:01:27.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.658+0000 7f385e932640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3858160150 con 0x7f38580d0e90 2024-05-11T01:01:27.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.658+0000 7f385e932640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3858160cf0 con 0x7f3858073250 2024-05-11T01:01:27.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.658+0000 7f3857fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f385815c3f0 0x7f38581605e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:59654/0 (socket says 172.21.15.52:59654) 2024-05-11T01:01:27.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.658+0000 7f3857fff640 1 -- 172.21.15.52:0/2650951627 learned_addr learned my addr 172.21.15.52:0/2650951627 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:27.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f38577fe640 1 --2- 172.21.15.52:0/2650951627 >> v2:172.21.15.119:3300/0 conn(0x7f38580d0e90 0x7f38580717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f385cecf640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3858073250 0x7f38580736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2649770731 0 0) 0x7f385815fe50 con 0x7f385815c3f0 2024-05-11T01:01:27.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3840003520 con 0x7f385815c3f0 2024-05-11T01:01:27.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3986872984 0 0) 0x7f3840003520 con 0x7f385815c3f0 2024-05-11T01:01:27.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3858073250 msgr2=0x7f38580736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3858073250 0x7f38580736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 >> v2:172.21.15.119:3300/0 conn(0x7f38580d0e90 msgr2=0x7f38580717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 --2- 172.21.15.52:0/2650951627 >> v2:172.21.15.119:3300/0 conn(0x7f38580d0e90 0x7f38580717a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38580d1570 con 0x7f385815c3f0 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f38480016f0 con 0x7f385815c3f0 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 >> v1:172.21.15.52:6789/0 conn(0x7f385815c3f0 legacy=0x7f38581605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3858073250 0x7f3840003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f3840004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3840004d40 0x7f38400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3858160150 con 0x7f3840003fd0 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3858160cf0 con 0x7f3840004d40 2024-05-11T01:01:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f384000ba10 con 0x7f3858073250 2024-05-11T01:01:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f38577fe640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f3840004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f3857fff640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3858073250 0x7f3840003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f385cecf640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3840004d40 0x7f38400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f38577fe640 1 -- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3858073250 msgr2=0x7f3840003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f38577fe640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3858073250 0x7f3840003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f38577fe640 1 -- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3840004d40 msgr2=0x7f38400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.659+0000 7f38577fe640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3840004d40 0x7f38400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.660+0000 7f38577fe640 1 -- 172.21.15.52:0/2650951627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3840003520 con 0x7f3840003fd0 2024-05-11T01:01:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.660+0000 7f38577fe640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f3840004800 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f384c009af0 tx=0x7f3840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5a7359e72fd67b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.660+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c036070 con 0x7f3840003fd0 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.660+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384c031400 con 0x7f3840003fd0 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.660+0000 7f3856ffd640 1 -- 172.21.15.52:0/2650951627 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c009690 con 0x7f3840003fd0 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.661+0000 7f385e932640 1 -- 172.21.15.52:0/2650951627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 msgr2=0x7f3840004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.661+0000 7f385e932640 1 --2- 172.21.15.52:0/2650951627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f3840004800 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f384c009af0 tx=0x7f3840005830 comp rx=0 tx=0).stop 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.661+0000 7f385cecf640 1 -- 172.21.15.52:0/2650951627 reap_dead start 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.661+0000 7f385e932640 1 -- 172.21.15.52:0/2650951627 shutdown_connections 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.661+0000 7f385e932640 1 -- 172.21.15.52:0/2650951627 >> 172.21.15.52:0/2650951627 conn(0x7f38580b3a30 msgr2=0x7f38580b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.661+0000 7f385e932640 1 -- 172.21.15.52:0/2650951627 shutdown_connections 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.661+0000 7f385e932640 1 -- 172.21.15.52:0/2650951627 wait complete. 2024-05-11T01:01:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.662+0000 7f385e932640 1 Processor -- start 2024-05-11T01:01:27.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.662+0000 7f385e932640 1 -- start start 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f385e932640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f38580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f385e932640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38580c8140 0x7f385815eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f385e932640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385814ff70 0x7f38580c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f385e932640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f38580d1b30 con 0x7f3840003fd0 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f385e932640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f38580d1310 con 0x7f38580c8140 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f385e932640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f38580736e0 con 0x7f385814ff70 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f3857fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f38580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f3857fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f38580b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43454/0 (socket says 172.21.15.52:43454) 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f3857fff640 1 -- 172.21.15.52:0/4195206574 learned_addr learned my addr 172.21.15.52:0/4195206574 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f3857fff640 1 -- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385814ff70 msgr2=0x7f38580c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f38577fe640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38580c8140 0x7f385815eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f3857fff640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385814ff70 0x7f38580c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.663+0000 7f3857fff640 1 -- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38580c8140 msgr2=0x7f385815eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.664+0000 7f3857fff640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38580c8140 0x7f385815eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.664+0000 7f3857fff640 1 -- 172.21.15.52:0/4195206574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3858150960 con 0x7f3840003fd0 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.664+0000 7f38577fe640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38580c8140 0x7f385815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.664+0000 7f3857fff640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f38580b4d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f38480072c0 tx=0x7f3848007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.664+0000 7f3854ff9640 1 -- 172.21.15.52:0/4195206574 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384800f940 con 0x7f3840003fd0 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.664+0000 7f3854ff9640 1 -- 172.21.15.52:0/4195206574 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3848018070 con 0x7f3840003fd0 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.664+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38580c8620 con 0x7f3840003fd0 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.664+0000 7f3854ff9640 1 -- 172.21.15.52:0/4195206574 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384800fd20 con 0x7f3840003fd0 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.664+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385815cbe0 con 0x7f3840003fd0 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.665+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3858069670 con 0x7f3840003fd0 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.665+0000 7f3854ff9640 1 -- 172.21.15.52:0/4195206574 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f384801d460 con 0x7f3840003fd0 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.666+0000 7f3854ff9640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3844076300 0x7f38440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.666+0000 7f3854ff9640 1 -- 172.21.15.52:0/4195206574 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3848097590 con 0x7f3840003fd0 2024-05-11T01:01:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.666+0000 7f38577fe640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3844076300 0x7f38440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.667+0000 7f38577fe640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3844076300 0x7f38440787c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f384c033000 tx=0x7f384c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.671+0000 7f3854ff9640 1 -- 172.21.15.52:0/4195206574 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3848060f80 con 0x7f3840003fd0 2024-05-11T01:01:27.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:27.936+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) -- 0x7f38580cd5a0 con 0x7f3840003fd0 2024-05-11T01:01:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.743+0000 7f3854ff9640 1 -- 172.21.15.52:0/4195206574 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v39) ==== 161+0+0 (secure 0 0 0) 0x7f3848060880 con 0x7f3840003fd0 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3844076300 msgr2=0x7f38440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3844076300 0x7f38440787c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f384c033000 tx=0x7f384c008030 comp rx=0 tx=0).stop 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 msgr2=0x7f38580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f38580b4d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f38480072c0 tx=0x7f3848007780 comp rx=0 tx=0).stop 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 shutdown_connections 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3844076300 0x7f38440787c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385814ff70 0x7f38580c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38580c8140 0x7f385815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 --2- 172.21.15.52:0/4195206574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3840003fd0 0x7f38580b4d90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.746+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 >> 172.21.15.52:0/4195206574 conn(0x7f38580b3a30 msgr2=0x7f38580b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.747+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 shutdown_connections 2024-05-11T01:01:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.747+0000 7f385e932640 1 -- 172.21.15.52:0/4195206574 wait complete. 2024-05-11T01:01:28.760 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-11T01:01:28.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.874+0000 7f59f6589640 1 Processor -- start 2024-05-11T01:01:28.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.875+0000 7f59f6589640 1 -- start start 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.875+0000 7f59f6589640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59f015f430 0x7f59f00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.875+0000 7f59f6589640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f59f00b5420 0x7f59f00d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.875+0000 7f59f6589640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 0x7f59f00d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.875+0000 7f59f6589640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f59f00d2a80 con 0x7f59f00b5420 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.875+0000 7f59f6589640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f59f00cfc60 con 0x7f59f015f430 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.875+0000 7f59f6589640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f59f0163120 con 0x7f59f00d1eb0 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59e7fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f59f00b5420 0x7f59f00d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59e7fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f59f00b5420 0x7f59f00d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43460/0 (socket says 172.21.15.52:43460) 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59e7fff640 1 -- 172.21.15.52:0/1565602741 learned_addr learned my addr 172.21.15.52:0/1565602741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59effff640 1 --2- 172.21.15.52:0/1565602741 >> v2:172.21.15.119:3300/0 conn(0x7f59f015f430 0x7f59f00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59f4b26640 1 --2- 172.21.15.52:0/1565602741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 0x7f59f00d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:28.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59e7fff640 1 -- 172.21.15.52:0/1565602741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 msgr2=0x7f59f00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59e7fff640 1 --2- 172.21.15.52:0/1565602741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 0x7f59f00d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59e7fff640 1 -- 172.21.15.52:0/1565602741 >> v2:172.21.15.119:3300/0 conn(0x7f59f015f430 msgr2=0x7f59f00b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59e7fff640 1 --2- 172.21.15.52:0/1565602741 >> v2:172.21.15.119:3300/0 conn(0x7f59f015f430 0x7f59f00b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59e7fff640 1 -- 172.21.15.52:0/1565602741 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f00d24c0 con 0x7f59f00b5420 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59effff640 1 --2- 172.21.15.52:0/1565602741 >> v2:172.21.15.119:3300/0 conn(0x7f59f015f430 0x7f59f00b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.876+0000 7f59f4b26640 1 --2- 172.21.15.52:0/1565602741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 0x7f59f00d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59e7fff640 1 --2- 172.21.15.52:0/1565602741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 0x7f59f00d1970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f59f00d6290 tx=0x7f59d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34013637a718cb2f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59ef7fe640 1 -- 172.21.15.52:0/1565602741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8037070 con 0x7f59f00b5420 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59ef7fe640 1 -- 172.21.15.52:0/1565602741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d8032440 con 0x7f59f00b5420 2024-05-11T01:01:28.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59ef7fe640 1 -- 172.21.15.52:0/1565602741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8032810 con 0x7f59f00b5420 2024-05-11T01:01:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59f6589640 1 -- 172.21.15.52:0/1565602741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 msgr2=0x7f59f00d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59f6589640 1 --2- 172.21.15.52:0/1565602741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 0x7f59f00d1970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f59f00d6290 tx=0x7f59d802a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59f6589640 1 -- 172.21.15.52:0/1565602741 shutdown_connections 2024-05-11T01:01:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59f6589640 1 --2- 172.21.15.52:0/1565602741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 0x7f59f00d1970 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59f6589640 1 --2- 172.21.15.52:0/1565602741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 0x7f59f00d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.877+0000 7f59f6589640 1 --2- 172.21.15.52:0/1565602741 >> v2:172.21.15.119:3300/0 conn(0x7f59f015f430 0x7f59f00b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.878+0000 7f59f6589640 1 -- 172.21.15.52:0/1565602741 >> 172.21.15.52:0/1565602741 conn(0x7f59f008ebb0 msgr2=0x7f59f009e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.878+0000 7f59f6589640 1 -- 172.21.15.52:0/1565602741 shutdown_connections 2024-05-11T01:01:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.878+0000 7f59f6589640 1 -- 172.21.15.52:0/1565602741 wait complete. 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.879+0000 7f59f6589640 1 Processor -- start 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.879+0000 7f59f6589640 1 -- start start 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.879+0000 7f59f6589640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 0x7f59f014e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.879+0000 7f59f6589640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 0x7f59f01f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.879+0000 7f59f6589640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f015f430 0x7f59f014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.879+0000 7f59f6589640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f59f00d01e0 con 0x7f59f00b5420 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59f6589640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59f00cff00 con 0x7f59f015f430 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59f6589640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f59f00d0c40 con 0x7f59f00d1eb0 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 0x7f59f014e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 0x7f59f014e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43474/0 (socket says 172.21.15.52:43474) 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59effff640 1 -- 172.21.15.52:0/2595561107 learned_addr learned my addr 172.21.15.52:0/2595561107 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59f4b26640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f015f430 0x7f59f014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59e7fff640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 0x7f59f01f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59effff640 1 -- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 msgr2=0x7f59f01f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59effff640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 0x7f59f01f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59effff640 1 -- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f015f430 msgr2=0x7f59f014c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59effff640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f015f430 0x7f59f014c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59effff640 1 -- 172.21.15.52:0/2595561107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f014f4f0 con 0x7f59f00b5420 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59f4b26640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f015f430 0x7f59f014c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59effff640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 0x7f59f014e590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59e000d800 tx=0x7f59e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59ed7fa640 1 -- 172.21.15.52:0/2595561107 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0004030 con 0x7f59f00b5420 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59ed7fa640 1 -- 172.21.15.52:0/2595561107 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e00041d0 con 0x7f59f00b5420 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f014ec50 con 0x7f59f00b5420 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59ed7fa640 1 -- 172.21.15.52:0/2595561107 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e000b6c0 con 0x7f59f00b5420 2024-05-11T01:01:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.880+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f014f210 con 0x7f59f00b5420 2024-05-11T01:01:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.881+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f0073f80 con 0x7f59f00b5420 2024-05-11T01:01:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.882+0000 7f59ed7fa640 1 -- 172.21.15.52:0/2595561107 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f59e000b860 con 0x7f59f00b5420 2024-05-11T01:01:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.883+0000 7f59ed7fa640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f59d0076300 0x7f59d00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.883+0000 7f59ed7fa640 1 -- 172.21.15.52:0/2595561107 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f59e0097770 con 0x7f59f00b5420 2024-05-11T01:01:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.883+0000 7f59e7fff640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f59d0076300 0x7f59d00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.883+0000 7f59e7fff640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f59d0076300 0x7f59d00787c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59d8034000 tx=0x7f59d802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:28.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:28.887+0000 7f59ed7fa640 1 -- 172.21.15.52:0/2595561107 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f59f0073f80 con 0x7f59f00b5420 2024-05-11T01:01:29.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.077+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f59f00695d0 con 0x7f59f00b5420 2024-05-11T01:01:29.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.081+0000 7f59ed7fa640 1 -- 172.21.15.52:0/2595561107 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f59e0061160 con 0x7f59f00b5420 2024-05-11T01:01:29.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f59d0076300 msgr2=0x7f59d00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f59d0076300 0x7f59d00787c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59d8034000 tx=0x7f59d802e000 comp rx=0 tx=0).stop 2024-05-11T01:01:29.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 msgr2=0x7f59f014e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 0x7f59f014e590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59e000d800 tx=0x7f59e000dcc0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 shutdown_connections 2024-05-11T01:01:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f59d0076300 0x7f59d00787c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59f015f430 0x7f59f014c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59f00d1eb0 0x7f59f01f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 --2- 172.21.15.52:0/2595561107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59f00b5420 0x7f59f014e590 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 >> 172.21.15.52:0/2595561107 conn(0x7f59f008ebb0 msgr2=0x7f59f00b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 shutdown_connections 2024-05-11T01:01:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.083+0000 7f59f6589640 1 -- 172.21.15.52:0/2595561107 wait complete. 2024-05-11T01:01:29.094 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T01:01:29.095 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T01:01:29.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T01:01:29.182 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T01:01:29.186 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:01:29.186 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:29.203 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:01:29.203 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:29.259 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:01:29.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.405+0000 7f2e658e3640 1 Processor -- start 2024-05-11T01:01:29.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.405+0000 7f2e658e3640 1 -- start start 2024-05-11T01:01:29.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.406+0000 7f2e658e3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e6006cd70 0x7f2e60073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.406+0000 7f2e658e3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e600cfd80 0x7f2e600d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.406+0000 7f2e658e3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e600d0230 0x7f2e600d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.406+0000 7f2e658e3640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6006d150 con 0x7f2e6006cd70 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.406+0000 7f2e658e3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6015fd30 con 0x7f2e600cfd80 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.406+0000 7f2e658e3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f2e60160010 con 0x7f2e600d0230 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.406+0000 7f2e5effd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e6006cd70 0x7f2e60073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.406+0000 7f2e5effd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e6006cd70 0x7f2e60073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43486/0 (socket says 172.21.15.52:43486) 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.406+0000 7f2e5effd640 1 -- 172.21.15.52:0/3868104770 learned_addr learned my addr 172.21.15.52:0/3868104770 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.407+0000 7f2e5e7fc640 1 --2- 172.21.15.52:0/3868104770 >> v2:172.21.15.119:3300/0 conn(0x7f2e600cfd80 0x7f2e600d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.407+0000 7f2e5effd640 1 -- 172.21.15.52:0/3868104770 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e600d0230 msgr2=0x7f2e600d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.407+0000 7f2e5effd640 1 --2- 172.21.15.52:0/3868104770 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e600d0230 0x7f2e600d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.407+0000 7f2e5effd640 1 -- 172.21.15.52:0/3868104770 >> v2:172.21.15.119:3300/0 conn(0x7f2e600cfd80 msgr2=0x7f2e600d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:29.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.407+0000 7f2e5effd640 1 --2- 172.21.15.52:0/3868104770 >> v2:172.21.15.119:3300/0 conn(0x7f2e600cfd80 0x7f2e600d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.407+0000 7f2e5effd640 1 -- 172.21.15.52:0/3868104770 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e601602f0 con 0x7f2e6006cd70 2024-05-11T01:01:29.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.407+0000 7f2e5effd640 1 --2- 172.21.15.52:0/3868104770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e6006cd70 0x7f2e60073230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e60074b50 tx=0x7f2e5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6244693b6bdff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:29.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e5dffb640 1 -- 172.21.15.52:0/3868104770 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58037070 con 0x7f2e6006cd70 2024-05-11T01:01:29.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e5dffb640 1 -- 172.21.15.52:0/3868104770 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e58032440 con 0x7f2e6006cd70 2024-05-11T01:01:29.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e5dffb640 1 -- 172.21.15.52:0/3868104770 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58032810 con 0x7f2e6006cd70 2024-05-11T01:01:29.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e658e3640 1 -- 172.21.15.52:0/3868104770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e6006cd70 msgr2=0x7f2e60073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e658e3640 1 --2- 172.21.15.52:0/3868104770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e6006cd70 0x7f2e60073230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e60074b50 tx=0x7f2e5802a140 comp rx=0 tx=0).stop 2024-05-11T01:01:29.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e658e3640 1 -- 172.21.15.52:0/3868104770 shutdown_connections 2024-05-11T01:01:29.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e658e3640 1 --2- 172.21.15.52:0/3868104770 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e600d0230 0x7f2e600d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e658e3640 1 --2- 172.21.15.52:0/3868104770 >> v2:172.21.15.119:3300/0 conn(0x7f2e600cfd80 0x7f2e600d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e658e3640 1 --2- 172.21.15.52:0/3868104770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e6006cd70 0x7f2e60073230 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e658e3640 1 -- 172.21.15.52:0/3868104770 >> 172.21.15.52:0/3868104770 conn(0x7f2e600b3af0 msgr2=0x7f2e600b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e658e3640 1 -- 172.21.15.52:0/3868104770 shutdown_connections 2024-05-11T01:01:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.408+0000 7f2e658e3640 1 -- 172.21.15.52:0/3868104770 wait complete. 2024-05-11T01:01:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.409+0000 7f2e658e3640 1 Processor -- start 2024-05-11T01:01:29.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.410+0000 7f2e658e3640 1 -- start start 2024-05-11T01:01:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.410+0000 7f2e658e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6006cd70 0x7f2e60142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.410+0000 7f2e658e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e600cfd80 0x7f2e600b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.410+0000 7f2e658e3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e600d0230 0x7f2e600b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.410+0000 7f2e658e3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6015fa60 con 0x7f2e600cfd80 2024-05-11T01:01:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.410+0000 7f2e658e3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6015f8e0 con 0x7f2e6006cd70 2024-05-11T01:01:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.410+0000 7f2e658e3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f2e60159120 con 0x7f2e600d0230 2024-05-11T01:01:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e600cfd80 0x7f2e600b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e600cfd80 0x7f2e600b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43494/0 (socket says 172.21.15.52:43494) 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5e7fc640 1 -- 172.21.15.52:0/1521140743 learned_addr learned my addr 172.21.15.52:0/1521140743 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5effd640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6006cd70 0x7f2e60142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5f7fe640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e600d0230 0x7f2e600b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5e7fc640 1 -- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e600d0230 msgr2=0x7f2e600b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5e7fc640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e600d0230 0x7f2e600b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5e7fc640 1 -- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6006cd70 msgr2=0x7f2e60142930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5e7fc640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6006cd70 0x7f2e60142930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5e7fc640 1 -- 172.21.15.52:0/1521140743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e600cf490 con 0x7f2e600cfd80 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e5e7fc640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e600cfd80 0x7f2e600b59d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800c7f0 tx=0x7f2e4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e57fff640 1 -- 172.21.15.52:0/1521140743 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48015070 con 0x7f2e600cfd80 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e57fff640 1 -- 172.21.15.52:0/1521140743 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e48004240 con 0x7f2e600cfd80 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e60142e70 con 0x7f2e600cfd80 2024-05-11T01:01:29.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e57fff640 1 -- 172.21.15.52:0/1521140743 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48007620 con 0x7f2e600cfd80 2024-05-11T01:01:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.411+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e60156fc0 con 0x7f2e600cfd80 2024-05-11T01:01:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.412+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e60069780 con 0x7f2e600cfd80 2024-05-11T01:01:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.413+0000 7f2e57fff640 1 -- 172.21.15.52:0/1521140743 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2e480077c0 con 0x7f2e600cfd80 2024-05-11T01:01:29.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.413+0000 7f2e57fff640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e400762d0 0x7f2e40078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.414+0000 7f2e57fff640 1 -- 172.21.15.52:0/1521140743 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e480976f0 con 0x7f2e600cfd80 2024-05-11T01:01:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.414+0000 7f2e5effd640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e400762d0 0x7f2e40078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.414+0000 7f2e5effd640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e400762d0 0x7f2e40078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2e58034000 tx=0x7f2e5802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:29.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.418+0000 7f2e57fff640 1 -- 172.21.15.52:0/1521140743 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2e60142e70 con 0x7f2e600cfd80 2024-05-11T01:01:29.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.624+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2e600ce490 con 0x7f2e600cfd80 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.624+0000 7f2e57fff640 1 -- 172.21.15.52:0/1521140743 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f2e600cf490 con 0x7f2e600cfd80 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e400762d0 msgr2=0x7f2e40078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e400762d0 0x7f2e40078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2e58034000 tx=0x7f2e5802e000 comp rx=0 tx=0).stop 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e600cfd80 msgr2=0x7f2e600b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e600cfd80 0x7f2e600b59d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800c7f0 tx=0x7f2e4800ccb0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 shutdown_connections 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f2e400762d0 0x7f2e40078790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e600d0230 0x7f2e600b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e600cfd80 0x7f2e600b59d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 --2- 172.21.15.52:0/1521140743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6006cd70 0x7f2e60142930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 >> 172.21.15.52:0/1521140743 conn(0x7f2e600b3af0 msgr2=0x7f2e600d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:29.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 shutdown_connections 2024-05-11T01:01:29.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.627+0000 7f2e658e3640 1 -- 172.21.15.52:0/1521140743 wait complete. 2024-05-11T01:01:29.638 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:01:29.638 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:29.655 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:01:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.799+0000 7fa3dc9b6640 1 Processor -- start 2024-05-11T01:01:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.799+0000 7fa3dc9b6640 1 -- start start 2024-05-11T01:01:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3dc9b6640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d80d2ea0 0x7fa3d80d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3dc9b6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa3d80b2600 0x7fa3d80b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3dc9b6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d80d1f90 con 0x7fa3d815f2d0 2024-05-11T01:01:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3dc9b6640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d80d2270 con 0x7fa3d80b2600 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3dc9b6640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d80d2550 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3d6d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa3d80b2600 0x7fa3d80b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3d6d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa3d80b2600 0x7fa3d80b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43500/0 (socket says 172.21.15.52:43500) 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3d6d9d640 1 -- 172.21.15.52:0/3817603022 learned_addr learned my addr 172.21.15.52:0/3817603022 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3cdd9b640 1 --2- 172.21.15.52:0/3817603022 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d80d2ea0 0x7fa3d80d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3d6d9d640 1 -- 172.21.15.52:0/3817603022 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d80d2ea0 msgr2=0x7fa3d80d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3d6d9d640 1 --2- 172.21.15.52:0/3817603022 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d80d2ea0 0x7fa3d80d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3d6d9d640 1 -- 172.21.15.52:0/3817603022 >> v1:172.21.15.119:6789/0 conn(0x7fa3d815f2d0 legacy=0x7fa3d806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3d6d9d640 1 -- 172.21.15.52:0/3817603022 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d80d2830 con 0x7fa3d80b2600 2024-05-11T01:01:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.800+0000 7fa3cdd9b640 1 --2- 172.21.15.52:0/3817603022 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d80d2ea0 0x7fa3d80d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.801+0000 7fa3d6d9d640 1 --2- 172.21.15.52:0/3817603022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80b2600 0x7fa3d80b2a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009e50 tx=0x7fa3d002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00e5d8b3e07ecf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.801+0000 7fa3d5d9b640 1 -- 172.21.15.52:0/3817603022 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0036070 con 0x7fa3d80b2600 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.801+0000 7fa3d5d9b640 1 -- 172.21.15.52:0/3817603022 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d0031600 con 0x7fa3d80b2600 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.801+0000 7fa3d5d9b640 1 -- 172.21.15.52:0/3817603022 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00319d0 con 0x7fa3d80b2600 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.802+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/3817603022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80b2600 msgr2=0x7fa3d80b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.802+0000 7fa3dc9b6640 1 --2- 172.21.15.52:0/3817603022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80b2600 0x7fa3d80b2a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009e50 tx=0x7fa3d002a710 comp rx=0 tx=0).stop 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.802+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/3817603022 shutdown_connections 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.802+0000 7fa3dc9b6640 1 --2- 172.21.15.52:0/3817603022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80b2600 0x7fa3d80b2a50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.802+0000 7fa3dc9b6640 1 --2- 172.21.15.52:0/3817603022 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d80d2ea0 0x7fa3d80d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.802+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/3817603022 >> 172.21.15.52:0/3817603022 conn(0x7fa3d809ba20 msgr2=0x7fa3d809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.802+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/3817603022 shutdown_connections 2024-05-11T01:01:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.802+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/3817603022 wait complete. 2024-05-11T01:01:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.803+0000 7fa3dc9b6640 1 Processor -- start 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.803+0000 7fa3dc9b6640 1 -- start start 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3dc9b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d80b2600 0x7fa3d814a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3dc9b6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80d2ea0 0x7fa3d8149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3dc9b6640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d815f2d0 0x7fa3d8150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3dc9b6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d815f830 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3dc9b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d815f6b0 con 0x7fa3d80b2600 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3dc9b6640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d80d1b80 con 0x7fa3d815f2d0 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3cdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80d2ea0 0x7fa3d8149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3cdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80d2ea0 0x7fa3d8149cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43506/0 (socket says 172.21.15.52:43506) 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3cdd9b640 1 -- 172.21.15.52:0/395623516 learned_addr learned my addr 172.21.15.52:0/395623516 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3d659c640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d80b2600 0x7fa3d814a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3d6d9d640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d815f2d0 0x7fa3d8150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3cdd9b640 1 -- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d815f2d0 msgr2=0x7fa3d8150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3cdd9b640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d815f2d0 0x7fa3d8150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3cdd9b640 1 -- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d80b2600 msgr2=0x7fa3d814a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3cdd9b640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d80b2600 0x7fa3d814a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3cdd9b640 1 -- 172.21.15.52:0/395623516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d8071bc0 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.804+0000 7fa3cdd9b640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80d2ea0 0x7fa3d8149cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000e730 tx=0x7fa3c000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.805+0000 7fa3cf7fe640 1 -- 172.21.15.52:0/395623516 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000f040 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.805+0000 7fa3cf7fe640 1 -- 172.21.15.52:0/395623516 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c0004230 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.805+0000 7fa3cf7fe640 1 -- 172.21.15.52:0/395623516 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000ee40 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.805+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d8160b00 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.805+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d8161080 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.806+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d8069780 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.806+0000 7fa3cf7fe640 1 -- 172.21.15.52:0/395623516 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa3c000c6d0 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.807+0000 7fa3cf7fe640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3b80762d0 0x7fa3b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.807+0000 7fa3cf7fe640 1 -- 172.21.15.52:0/395623516 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3c0097220 con 0x7fa3d80d2ea0 2024-05-11T01:01:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.808+0000 7fa3d659c640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3b80762d0 0x7fa3b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.808+0000 7fa3d659c640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3b80762d0 0x7fa3b8078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80015e0 tx=0x7fa3c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:29.812+0000 7fa3cf7fe640 1 -- 172.21.15.52:0/395623516 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa3c0017020 con 0x7fa3d80d2ea0 2024-05-11T01:01:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.012+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa3d80b1b60 con 0x7fa3d80d2ea0 2024-05-11T01:01:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.013+0000 7fa3cf7fe640 1 -- 172.21.15.52:0/395623516 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fa3c0060c10 con 0x7fa3d80d2ea0 2024-05-11T01:01:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:01:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.014+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3b80762d0 msgr2=0x7fa3b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.014+0000 7fa3dc9b6640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3b80762d0 0x7fa3b8078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80015e0 tx=0x7fa3c8009040 comp rx=0 tx=0).stop 2024-05-11T01:01:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.014+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80d2ea0 msgr2=0x7fa3d8149cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.014+0000 7fa3dc9b6640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80d2ea0 0x7fa3d8149cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000e730 tx=0x7fa3c000ebf0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.015+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 shutdown_connections 2024-05-11T01:01:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.015+0000 7fa3dc9b6640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa3b80762d0 0x7fa3b8078790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.015+0000 7fa3dc9b6640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3d815f2d0 0x7fa3d8150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.015+0000 7fa3dc9b6640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3d80d2ea0 0x7fa3d8149cc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.015+0000 7fa3dc9b6640 1 --2- 172.21.15.52:0/395623516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d80b2600 0x7fa3d814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.015+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 >> 172.21.15.52:0/395623516 conn(0x7fa3d809ba20 msgr2=0x7fa3d80d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.015+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 shutdown_connections 2024-05-11T01:01:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.015+0000 7fa3dc9b6640 1 -- 172.21.15.52:0/395623516 wait complete. 2024-05-11T01:01:30.026 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-05-11T01:01:30.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:30.026 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:30.052 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:01:30.052 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:01:30.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:30.052 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T01:01:30.114 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:01:30.114 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:30.134 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:01:30.134 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:30.195 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:01:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.299+0000 7f6541d6a640 1 Processor -- start 2024-05-11T01:01:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.299+0000 7f6541d6a640 1 -- start start 2024-05-11T01:01:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f6541d6a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f653c0afb30 0x7f653c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f6541d6a640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653c0aea90 con 0x7f653c073600 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f6541d6a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653c0aed70 con 0x7f653c072ce0 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f6541d6a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f653c06e760 con 0x7f653c0afb30 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f653affd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f653c0afb30 0x7f653c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f653bfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f653c073600 0x7f653c165b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:59666/0 (socket says 172.21.15.52:59666) 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f653bfff640 1 -- 172.21.15.52:0/3076202139 learned_addr learned my addr 172.21.15.52:0/3076202139 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f653affd640 1 -- 172.21.15.52:0/3076202139 >> v1:172.21.15.119:6789/0 conn(0x7f653c072ce0 legacy=0x7f653c06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f653affd640 1 -- 172.21.15.52:0/3076202139 >> v1:172.21.15.52:6789/0 conn(0x7f653c073600 legacy=0x7f653c165b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f653a7fc640 1 -- 172.21.15.52:0/3076202139 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3853009579 0 0) 0x7f653c0aea90 con 0x7f653c073600 2024-05-11T01:01:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.300+0000 7f653affd640 1 -- 172.21.15.52:0/3076202139 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c0af050 con 0x7f653c0afb30 2024-05-11T01:01:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.301+0000 7f653affd640 1 --2- 172.21.15.52:0/3076202139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 0x7f653c0730c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65300097a0 tx=0x7f653002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df69aa9726da4bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.301+0000 7f653a7fc640 1 -- 172.21.15.52:0/3076202139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530037070 con 0x7f653c0afb30 2024-05-11T01:01:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.301+0000 7f653a7fc640 1 -- 172.21.15.52:0/3076202139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6530032440 con 0x7f653c0afb30 2024-05-11T01:01:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.301+0000 7f653a7fc640 1 -- 172.21.15.52:0/3076202139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530032810 con 0x7f653c0afb30 2024-05-11T01:01:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.302+0000 7f6541d6a640 1 -- 172.21.15.52:0/3076202139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 msgr2=0x7f653c0730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.302+0000 7f6541d6a640 1 --2- 172.21.15.52:0/3076202139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 0x7f653c0730c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65300097a0 tx=0x7f653002a180 comp rx=0 tx=0).stop 2024-05-11T01:01:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.302+0000 7f6541d6a640 1 -- 172.21.15.52:0/3076202139 shutdown_connections 2024-05-11T01:01:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.302+0000 7f6541d6a640 1 --2- 172.21.15.52:0/3076202139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 0x7f653c0730c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.302+0000 7f6541d6a640 1 -- 172.21.15.52:0/3076202139 >> 172.21.15.52:0/3076202139 conn(0x7f653c0b2a00 msgr2=0x7f653c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.302+0000 7f6541d6a640 1 -- 172.21.15.52:0/3076202139 shutdown_connections 2024-05-11T01:01:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.302+0000 7f6541d6a640 1 -- 172.21.15.52:0/3076202139 wait complete. 2024-05-11T01:01:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.303+0000 7f6541d6a640 1 Processor -- start 2024-05-11T01:01:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.303+0000 7f6541d6a640 1 -- start start 2024-05-11T01:01:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f6541d6a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f653c072ce0 0x7f653c1e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f6541d6a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f653c073600 0x7f653c14c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f6541d6a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 0x7f653c09d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f6541d6a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f653c0af610 con 0x7f653c0afb30 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f6541d6a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f653c073a30 con 0x7f653c072ce0 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f6541d6a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f653c1e5790 con 0x7f653c073600 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 0x7f653c09d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 0x7f653c09d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43526/0 (socket says 172.21.15.52:43526) 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653bfff640 1 -- 172.21.15.52:0/2382266807 learned_addr learned my addr 172.21.15.52:0/2382266807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653b7fe640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f653c072ce0 0x7f653c1e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653affd640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f653c073600 0x7f653c14c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653bfff640 1 -- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f653c073600 msgr2=0x7f653c14c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653bfff640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f653c073600 0x7f653c14c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653bfff640 1 -- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f653c072ce0 msgr2=0x7f653c1e4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653bfff640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f653c072ce0 0x7f653c1e4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.304+0000 7f653bfff640 1 -- 172.21.15.52:0/2382266807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c1e5a70 con 0x7f653c0afb30 2024-05-11T01:01:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.305+0000 7f653bfff640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 0x7f653c09d010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f653400c7a0 tx=0x7f653400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.305+0000 7f6523fff640 1 -- 172.21.15.52:0/2382266807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534018070 con 0x7f653c0afb30 2024-05-11T01:01:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.305+0000 7f6523fff640 1 -- 172.21.15.52:0/2382266807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653400a5e0 con 0x7f653c0afb30 2024-05-11T01:01:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.305+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653c1e5d50 con 0x7f653c0afb30 2024-05-11T01:01:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.305+0000 7f6523fff640 1 -- 172.21.15.52:0/2382266807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400aa50 con 0x7f653c0afb30 2024-05-11T01:01:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.305+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f653c1e6310 con 0x7f653c0afb30 2024-05-11T01:01:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.306+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653c1e54e0 con 0x7f653c0afb30 2024-05-11T01:01:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.306+0000 7f6523fff640 1 -- 172.21.15.52:0/2382266807 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f653401b070 con 0x7f653c0afb30 2024-05-11T01:01:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.307+0000 7f6523fff640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f65280762d0 0x7f6528078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.307+0000 7f6523fff640 1 -- 172.21.15.52:0/2382266807 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6534096ad0 con 0x7f653c0afb30 2024-05-11T01:01:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.307+0000 7f653b7fe640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f65280762d0 0x7f6528078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.308+0000 7f653b7fe640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f65280762d0 0x7f6528078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f652c004f60 tx=0x7f652c007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.311+0000 7f6523fff640 1 -- 172.21.15.52:0/2382266807 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f653405fde0 con 0x7f653c0afb30 2024-05-11T01:01:30.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.516+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f653c069780 con 0x7f653c0afb30 2024-05-11T01:01:30.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.517+0000 7f6523fff640 1 -- 172.21.15.52:0/2382266807 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f65340640e0 con 0x7f653c0afb30 2024-05-11T01:01:30.517 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:01:30.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f65280762d0 msgr2=0x7f6528078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f65280762d0 0x7f6528078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f652c004f60 tx=0x7f652c007e30 comp rx=0 tx=0).stop 2024-05-11T01:01:30.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 msgr2=0x7f653c09d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 0x7f653c09d010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f653400c7a0 tx=0x7f653400cc60 comp rx=0 tx=0).stop 2024-05-11T01:01:30.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 shutdown_connections 2024-05-11T01:01:30.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f65280762d0 0x7f6528078790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f653c0afb30 0x7f653c09d010 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f653c073600 0x7f653c14c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 --2- 172.21.15.52:0/2382266807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f653c072ce0 0x7f653c1e4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 >> 172.21.15.52:0/2382266807 conn(0x7f653c0b2a00 msgr2=0x7f653c0b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:30.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 shutdown_connections 2024-05-11T01:01:30.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.519+0000 7f6541d6a640 1 -- 172.21.15.52:0/2382266807 wait complete. 2024-05-11T01:01:30.530 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:01:30.530 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:30.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:01:30.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.654+0000 7ffaecd5f640 1 Processor -- start 2024-05-11T01:01:30.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.654+0000 7ffaecd5f640 1 -- start start 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.655+0000 7ffaecd5f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffae8160480 0x7ffae80d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.655+0000 7ffaecd5f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 0x7ffae80d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.655+0000 7ffaecd5f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffae8163250 con 0x7ffae815f430 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.655+0000 7ffaecd5f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ffae8163530 con 0x7ffae8160480 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.655+0000 7ffaecd5f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffae8163810 con 0x7ffae80d32c0 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.656+0000 7ffae5d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffae8160480 0x7ffae80d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.656+0000 7ffae5d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffae8160480 0x7ffae80d2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43530/0 (socket says 172.21.15.52:43530) 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.656+0000 7ffae5d9b640 1 -- 172.21.15.52:0/3391670976 learned_addr learned my addr 172.21.15.52:0/3391670976 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.656+0000 7ffae6d9d640 1 --2- 172.21.15.52:0/3391670976 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 0x7ffae80d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.656+0000 7ffae5d9b640 1 -- 172.21.15.52:0/3391670976 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 msgr2=0x7ffae80d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.656+0000 7ffae5d9b640 1 --2- 172.21.15.52:0/3391670976 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 0x7ffae80d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.656+0000 7ffae5d9b640 1 -- 172.21.15.52:0/3391670976 >> v1:172.21.15.119:6789/0 conn(0x7ffae815f430 legacy=0x7ffae815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.656+0000 7ffae5d9b640 1 -- 172.21.15.52:0/3391670976 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae8163af0 con 0x7ffae8160480 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.657+0000 7ffae5d9b640 1 --2- 172.21.15.52:0/3391670976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 0x7ffae80d2d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009870 tx=0x7ffad002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27633582a70feaec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.657+0000 7ffae559a640 1 -- 172.21.15.52:0/3391670976 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0037070 con 0x7ffae8160480 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.657+0000 7ffae559a640 1 -- 172.21.15.52:0/3391670976 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad0032440 con 0x7ffae8160480 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.657+0000 7ffae559a640 1 -- 172.21.15.52:0/3391670976 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0032810 con 0x7ffae8160480 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.657+0000 7ffaecd5f640 1 -- 172.21.15.52:0/3391670976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 msgr2=0x7ffae80d2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.657+0000 7ffaecd5f640 1 --2- 172.21.15.52:0/3391670976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 0x7ffae80d2d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009870 tx=0x7ffad002a270 comp rx=0 tx=0).stop 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.658+0000 7ffaecd5f640 1 -- 172.21.15.52:0/3391670976 shutdown_connections 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.658+0000 7ffaecd5f640 1 --2- 172.21.15.52:0/3391670976 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 0x7ffae80d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.658+0000 7ffaecd5f640 1 --2- 172.21.15.52:0/3391670976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 0x7ffae80d2d80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.658+0000 7ffaecd5f640 1 -- 172.21.15.52:0/3391670976 >> 172.21.15.52:0/3391670976 conn(0x7ffae808eb70 msgr2=0x7ffae809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.658+0000 7ffaecd5f640 1 -- 172.21.15.52:0/3391670976 shutdown_connections 2024-05-11T01:01:30.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.658+0000 7ffaecd5f640 1 -- 172.21.15.52:0/3391670976 wait complete. 2024-05-11T01:01:30.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.659+0000 7ffaecd5f640 1 Processor -- start 2024-05-11T01:01:30.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.659+0000 7ffaecd5f640 1 -- start start 2024-05-11T01:01:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffaecd5f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 0x7ffae8157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffaecd5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae815f430 0x7ffae80ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffaecd5f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 0x7ffae80ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffaecd5f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8163030 con 0x7ffae8160480 2024-05-11T01:01:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffaecd5f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8162eb0 con 0x7ffae815f430 2024-05-11T01:01:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffaecd5f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffae81608f0 con 0x7ffae80d32c0 2024-05-11T01:01:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 0x7ffae80ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 0x7ffae80ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43534/0 (socket says 172.21.15.52:43534) 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae6d9d640 1 -- 172.21.15.52:0/2814890547 learned_addr learned my addr 172.21.15.52:0/2814890547 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae5d9b640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae815f430 0x7ffae80ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae659c640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 0x7ffae8157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae6d9d640 1 -- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 msgr2=0x7ffae8157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae6d9d640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 0x7ffae8157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae6d9d640 1 -- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae815f430 msgr2=0x7ffae80ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae6d9d640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae815f430 0x7ffae80ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.660+0000 7ffae6d9d640 1 -- 172.21.15.52:0/2814890547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae80cef50 con 0x7ffae8160480 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.661+0000 7ffae6d9d640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 0x7ffae80ce570 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffad8007e40 tx=0x7ffad800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.661+0000 7ffadeffd640 1 -- 172.21.15.52:0/2814890547 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8014620 con 0x7ffae8160480 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.661+0000 7ffadeffd640 1 -- 172.21.15.52:0/2814890547 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad8019070 con 0x7ffae8160480 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.661+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae80cc210 con 0x7ffae8160480 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.661+0000 7ffadeffd640 1 -- 172.21.15.52:0/2814890547 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8014a20 con 0x7ffae8160480 2024-05-11T01:01:30.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.661+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae80cc7d0 con 0x7ffae8160480 2024-05-11T01:01:30.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.662+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae8069780 con 0x7ffae8160480 2024-05-11T01:01:30.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.662+0000 7ffadeffd640 1 -- 172.21.15.52:0/2814890547 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffad8014bc0 con 0x7ffae8160480 2024-05-11T01:01:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.663+0000 7ffadeffd640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffac8076300 0x7ffac80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.663+0000 7ffadeffd640 1 -- 172.21.15.52:0/2814890547 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffad8097cb0 con 0x7ffae8160480 2024-05-11T01:01:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.663+0000 7ffae659c640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffac8076300 0x7ffac80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.664+0000 7ffae659c640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffac8076300 0x7ffac80787c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffae80cf740 tx=0x7ffae0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:30.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.667+0000 7ffadeffd640 1 -- 172.21.15.52:0/2814890547 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffad8061050 con 0x7ffae8160480 2024-05-11T01:01:30.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.871+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffae80b5360 con 0x7ffae8160480 2024-05-11T01:01:30.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.871+0000 7ffadeffd640 1 -- 172.21.15.52:0/2814890547 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7ffad8065350 con 0x7ffae8160480 2024-05-11T01:01:30.872 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:01:30.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffac8076300 msgr2=0x7ffac80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffac8076300 0x7ffac80787c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffae80cf740 tx=0x7ffae0009040 comp rx=0 tx=0).stop 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 msgr2=0x7ffae80ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 0x7ffae80ce570 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffad8007e40 tx=0x7ffad800e4b0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 shutdown_connections 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7ffac8076300 0x7ffac80787c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffae8160480 0x7ffae80ce570 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffae815f430 0x7ffae80ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 --2- 172.21.15.52:0/2814890547 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffae80d32c0 0x7ffae8157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 >> 172.21.15.52:0/2814890547 conn(0x7ffae808eb70 msgr2=0x7ffae80d3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 shutdown_connections 2024-05-11T01:01:30.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:30.873+0000 7ffaecd5f640 1 -- 172.21.15.52:0/2814890547 wait complete. 2024-05-11T01:01:30.885 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi119.front.sepia.ceph.com... 2024-05-11T01:01:30.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:30.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-11T01:01:30.916 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:01:30.917 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:01:30.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:30.917 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T01:01:30.994 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:01:30.994 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:31.010 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:01:31.011 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:31.067 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:01:31.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.212+0000 7f6d22bc8640 1 Processor -- start 2024-05-11T01:01:31.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.212+0000 7f6d22bc8640 1 -- start start 2024-05-11T01:01:31.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d22bc8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6d1c0d3330 0x7f6d1c0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d22bc8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d1c0d1f90 con 0x7f6d1c0d2ea0 2024-05-11T01:01:31.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d22bc8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d1c0d2270 con 0x7f6d1c15f2d0 2024-05-11T01:01:31.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d22bc8640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f6d1c0d2550 con 0x7f6d1c0d3330 2024-05-11T01:01:31.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d21165640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6d1c0d3330 0x7f6d1c0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d21165640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6d1c0d3330 0x7f6d1c0b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43544/0 (socket says 172.21.15.52:43544) 2024-05-11T01:01:31.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d13fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6d1c0d2ea0 0x7f6d1c0b2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:59676/0 (socket says 172.21.15.52:59676) 2024-05-11T01:01:31.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d21165640 1 -- 172.21.15.52:0/1837963002 learned_addr learned my addr 172.21.15.52:0/1837963002 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:31.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d21165640 1 -- 172.21.15.52:0/1837963002 >> v1:172.21.15.119:6789/0 conn(0x7f6d1c15f2d0 legacy=0x7f6d1c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d21165640 1 -- 172.21.15.52:0/1837963002 >> v1:172.21.15.52:6789/0 conn(0x7f6d1c0d2ea0 legacy=0x7f6d1c0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d21165640 1 -- 172.21.15.52:0/1837963002 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1c0d2830 con 0x7f6d1c0d3330 2024-05-11T01:01:31.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.213+0000 7f6d137fe640 1 -- 172.21.15.52:0/1837963002 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1793809898 0 0) 0x7f6d1c0d1f90 con 0x7f6d1c0d2ea0 2024-05-11T01:01:31.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.214+0000 7f6d21165640 1 --2- 172.21.15.52:0/1837963002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 0x7f6d1c0b2d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009e50 tx=0x7f6d1402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159b4d8637909432 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:31.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.214+0000 7f6d137fe640 1 -- 172.21.15.52:0/1837963002 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14036070 con 0x7f6d1c0d3330 2024-05-11T01:01:31.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.214+0000 7f6d137fe640 1 -- 172.21.15.52:0/1837963002 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d14031600 con 0x7f6d1c0d3330 2024-05-11T01:01:31.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.214+0000 7f6d137fe640 1 -- 172.21.15.52:0/1837963002 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d140319f0 con 0x7f6d1c0d3330 2024-05-11T01:01:31.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.215+0000 7f6d22bc8640 1 -- 172.21.15.52:0/1837963002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 msgr2=0x7f6d1c0b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.215+0000 7f6d22bc8640 1 --2- 172.21.15.52:0/1837963002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 0x7f6d1c0b2d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009e50 tx=0x7f6d1402a7c0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.215+0000 7f6d22bc8640 1 -- 172.21.15.52:0/1837963002 shutdown_connections 2024-05-11T01:01:31.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.215+0000 7f6d22bc8640 1 --2- 172.21.15.52:0/1837963002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 0x7f6d1c0b2d10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.215+0000 7f6d22bc8640 1 -- 172.21.15.52:0/1837963002 >> 172.21.15.52:0/1837963002 conn(0x7f6d1c09ba20 msgr2=0x7f6d1c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:31.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.215+0000 7f6d22bc8640 1 -- 172.21.15.52:0/1837963002 shutdown_connections 2024-05-11T01:01:31.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.215+0000 7f6d22bc8640 1 -- 172.21.15.52:0/1837963002 wait complete. 2024-05-11T01:01:31.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.216+0000 7f6d22bc8640 1 Processor -- start 2024-05-11T01:01:31.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.216+0000 7f6d22bc8640 1 -- start start 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d22bc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0d2ea0 0x7f6d1c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d22bc8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 0x7f6d1c161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d22bc8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6d1c15f2d0 0x7f6d1c14af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d22bc8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c15f6b0 con 0x7f6d1c0d3330 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d22bc8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c0d3740 con 0x7f6d1c0d2ea0 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d22bc8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6d1c15f830 con 0x7f6d1c15f2d0 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d13fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 0x7f6d1c161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d13fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 0x7f6d1c161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43546/0 (socket says 172.21.15.52:43546) 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d13fff640 1 -- 172.21.15.52:0/2095477572 learned_addr learned my addr 172.21.15.52:0/2095477572 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d20964640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0d2ea0 0x7f6d1c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d21165640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6d1c15f2d0 0x7f6d1c14af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d13fff640 1 -- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6d1c15f2d0 msgr2=0x7f6d1c14af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d13fff640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6d1c15f2d0 0x7f6d1c14af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d13fff640 1 -- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0d2ea0 msgr2=0x7f6d1c155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d13fff640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0d2ea0 0x7f6d1c155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d13fff640 1 -- 172.21.15.52:0/2095477572 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1c15b4e0 con 0x7f6d1c0d3330 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.217+0000 7f6d13fff640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 0x7f6d1c161150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c450 tx=0x7f6d0c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.218+0000 7f6d117fa640 1 -- 172.21.15.52:0/2095477572 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c018070 con 0x7f6d1c0d3330 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.218+0000 7f6d117fa640 1 -- 172.21.15.52:0/2095477572 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c0074e0 con 0x7f6d1c0d3330 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.218+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1c156660 con 0x7f6d1c0d3330 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.218+0000 7f6d117fa640 1 -- 172.21.15.52:0/2095477572 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c004a20 con 0x7f6d1c0d3330 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.218+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d1c155db0 con 0x7f6d1c0d3330 2024-05-11T01:01:31.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.219+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d1c06da30 con 0x7f6d1c0d3330 2024-05-11T01:01:31.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.219+0000 7f6d117fa640 1 -- 172.21.15.52:0/2095477572 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6d0c01b070 con 0x7f6d1c0d3330 2024-05-11T01:01:31.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.220+0000 7f6d117fa640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6d080762d0 0x7f6d08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.220+0000 7f6d117fa640 1 -- 172.21.15.52:0/2095477572 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6d0c0967a0 con 0x7f6d1c0d3330 2024-05-11T01:01:31.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.220+0000 7f6d20964640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6d080762d0 0x7f6d08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.221+0000 7f6d20964640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6d080762d0 0x7f6d08078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d18000f60 tx=0x7f6d18009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:31.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.224+0000 7f6d117fa640 1 -- 172.21.15.52:0/2095477572 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6d1c06da30 con 0x7f6d1c0d3330 2024-05-11T01:01:31.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.428+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6d1c069780 con 0x7f6d1c0d3330 2024-05-11T01:01:31.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.428+0000 7f6d117fa640 1 -- 172.21.15.52:0/2095477572 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f6d0c05fc00 con 0x7f6d1c0d3330 2024-05-11T01:01:31.428 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:01:31.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6d080762d0 msgr2=0x7f6d08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6d080762d0 0x7f6d08078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d18000f60 tx=0x7f6d18009040 comp rx=0 tx=0).stop 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 msgr2=0x7f6d1c161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 0x7f6d1c161150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c450 tx=0x7f6d0c00c910 comp rx=0 tx=0).stop 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 shutdown_connections 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6d080762d0 0x7f6d08078790 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6d1c15f2d0 0x7f6d1c14af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d1c0d3330 0x7f6d1c161150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 --2- 172.21.15.52:0/2095477572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0d2ea0 0x7f6d1c155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 >> 172.21.15.52:0/2095477572 conn(0x7f6d1c09ba20 msgr2=0x7f6d1c0b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 shutdown_connections 2024-05-11T01:01:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.430+0000 7f6d22bc8640 1 -- 172.21.15.52:0/2095477572 wait complete. 2024-05-11T01:01:31.442 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:01:31.442 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:31.459 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:01:31.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.604+0000 7f7fdfc07640 1 Processor -- start 2024-05-11T01:01:31.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.604+0000 7f7fdfc07640 1 -- start start 2024-05-11T01:01:31.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdfc07640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7fd815f2d0 0x7f7fd806e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdfc07640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7fd806e5d0 0x7f7fd80d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdfc07640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd80d33e0 0x7f7fd80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdfc07640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd80d37f0 con 0x7f7fd806e5d0 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdfc07640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd80d1f90 con 0x7f7fd815f2d0 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdfc07640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd80d2270 con 0x7f7fd80d33e0 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdd1a2640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7fd806e5d0 0x7f7fd80d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdd1a2640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7fd806e5d0 0x7f7fd80d2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43550/0 (socket says 172.21.15.52:43550) 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdd1a2640 1 -- 172.21.15.52:0/3007296528 learned_addr learned my addr 172.21.15.52:0/3007296528 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fde1a4640 1 --2- 172.21.15.52:0/3007296528 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd80d33e0 0x7f7fd80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdd9a3640 1 --2- 172.21.15.52:0/3007296528 >> v2:172.21.15.119:3300/0 conn(0x7f7fd815f2d0 0x7f7fd806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdd1a2640 1 -- 172.21.15.52:0/3007296528 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd80d33e0 msgr2=0x7f7fd80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdd1a2640 1 --2- 172.21.15.52:0/3007296528 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd80d33e0 0x7f7fd80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdd1a2640 1 -- 172.21.15.52:0/3007296528 >> v2:172.21.15.119:3300/0 conn(0x7f7fd815f2d0 msgr2=0x7f7fd806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:31.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdd1a2640 1 --2- 172.21.15.52:0/3007296528 >> v2:172.21.15.119:3300/0 conn(0x7f7fd815f2d0 0x7f7fd806e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.605+0000 7f7fdd1a2640 1 -- 172.21.15.52:0/3007296528 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd80d2550 con 0x7f7fd806e5d0 2024-05-11T01:01:31.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.606+0000 7f7fdd1a2640 1 --2- 172.21.15.52:0/3007296528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd806e5d0 0x7f7fd80d2ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80746a0 tx=0x7f7fc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6455754c3bbb7b21 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:31.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.606+0000 7f7fdc9a1640 1 -- 172.21.15.52:0/3007296528 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8037070 con 0x7f7fd806e5d0 2024-05-11T01:01:31.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.606+0000 7f7fdc9a1640 1 -- 172.21.15.52:0/3007296528 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc8032440 con 0x7f7fd806e5d0 2024-05-11T01:01:31.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.606+0000 7f7fdc9a1640 1 -- 172.21.15.52:0/3007296528 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8032810 con 0x7f7fd806e5d0 2024-05-11T01:01:31.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.607+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3007296528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd806e5d0 msgr2=0x7f7fd80d2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.607+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3007296528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd806e5d0 0x7f7fd80d2ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80746a0 tx=0x7f7fc802a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.607+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3007296528 shutdown_connections 2024-05-11T01:01:31.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.607+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3007296528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd806e5d0 0x7f7fd80d2ea0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.607+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3007296528 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd80d33e0 0x7f7fd80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.607+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3007296528 >> v2:172.21.15.119:3300/0 conn(0x7f7fd815f2d0 0x7f7fd806e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.607+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3007296528 >> 172.21.15.52:0/3007296528 conn(0x7f7fd809ba20 msgr2=0x7f7fd809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:31.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.607+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3007296528 shutdown_connections 2024-05-11T01:01:31.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.607+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3007296528 wait complete. 2024-05-11T01:01:31.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.608+0000 7f7fdfc07640 1 Processor -- start 2024-05-11T01:01:31.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.608+0000 7f7fdfc07640 1 -- start start 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdfc07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd806e5d0 0x7f7fd81482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdfc07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd80d33e0 0x7f7fd8157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdfc07640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd815f2d0 0x7f7fd80b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdfc07640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd815f6b0 con 0x7f7fd80d33e0 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdfc07640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd80d2b10 con 0x7f7fd806e5d0 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdfc07640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd815f830 con 0x7f7fd815f2d0 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd1a2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd80d33e0 0x7f7fd8157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd1a2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd80d33e0 0x7f7fd8157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43560/0 (socket says 172.21.15.52:43560) 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd1a2640 1 -- 172.21.15.52:0/3524804350 learned_addr learned my addr 172.21.15.52:0/3524804350 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fde1a4640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd815f2d0 0x7f7fd80b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd9a3640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd806e5d0 0x7f7fd81482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd1a2640 1 -- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd815f2d0 msgr2=0x7f7fd80b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd1a2640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd815f2d0 0x7f7fd80b7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd1a2640 1 -- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd806e5d0 msgr2=0x7f7fd81482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd1a2640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd806e5d0 0x7f7fd81482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd1a2640 1 -- 172.21.15.52:0/3524804350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd80b6b10 con 0x7f7fd80d33e0 2024-05-11T01:01:31.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.609+0000 7f7fdd1a2640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd80d33e0 0x7f7fd8157cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7fc802a220 tx=0x7f7fc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:31.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.610+0000 7f7fce7fc640 1 -- 172.21.15.52:0/3524804350 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8041020 con 0x7f7fd80d33e0 2024-05-11T01:01:31.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.610+0000 7f7fce7fc640 1 -- 172.21.15.52:0/3524804350 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc80306c0 con 0x7f7fd80d33e0 2024-05-11T01:01:31.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.610+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd80b6db0 con 0x7f7fd80d33e0 2024-05-11T01:01:31.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.610+0000 7f7fce7fc640 1 -- 172.21.15.52:0/3524804350 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8037040 con 0x7f7fd80d33e0 2024-05-11T01:01:31.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.610+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd80b7370 con 0x7f7fd80d33e0 2024-05-11T01:01:31.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.611+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd8069780 con 0x7f7fd80d33e0 2024-05-11T01:01:31.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.611+0000 7f7fce7fc640 1 -- 172.21.15.52:0/3524804350 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7fc8044020 con 0x7f7fd80d33e0 2024-05-11T01:01:31.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.612+0000 7f7fce7fc640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7fbc0762d0 0x7f7fbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:31.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.612+0000 7f7fce7fc640 1 -- 172.21.15.52:0/3524804350 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7fc80b7e40 con 0x7f7fd80d33e0 2024-05-11T01:01:31.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.612+0000 7f7fdd9a3640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7fbc0762d0 0x7f7fbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:31.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.613+0000 7f7fdd9a3640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7fbc0762d0 0x7f7fbc078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006fd0 tx=0x7f7fd4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:31.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.616+0000 7f7fce7fc640 1 -- 172.21.15.52:0/3524804350 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7fc804b030 con 0x7f7fd80d33e0 2024-05-11T01:01:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.819+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7fd80b1aa0 con 0x7f7fd80d33e0 2024-05-11T01:01:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.819+0000 7f7fce7fc640 1 -- 172.21.15.52:0/3524804350 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f7fc80817a0 con 0x7f7fd80d33e0 2024-05-11T01:01:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:01:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7fbc0762d0 msgr2=0x7f7fbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7fbc0762d0 0x7f7fbc078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006fd0 tx=0x7f7fd4008040 comp rx=0 tx=0).stop 2024-05-11T01:01:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd80d33e0 msgr2=0x7f7fd8157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd80d33e0 0x7f7fd8157cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7fc802a220 tx=0x7f7fc80046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 shutdown_connections 2024-05-11T01:01:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f7fbc0762d0 0x7f7fbc078790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7fd815f2d0 0x7f7fd80b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd80d33e0 0x7f7fd8157cc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 --2- 172.21.15.52:0/3524804350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd806e5d0 0x7f7fd81482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 >> 172.21.15.52:0/3524804350 conn(0x7f7fd809ba20 msgr2=0x7f7fd80b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 shutdown_connections 2024-05-11T01:01:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:31.821+0000 7f7fdfc07640 1 -- 172.21.15.52:0/3524804350 wait complete. 2024-05-11T01:01:31.832 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-05-11T01:01:31.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:31.832 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-11T01:01:31.858 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:01:31.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:01:31.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:31.859 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-11T01:01:31.915 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:01:31.916 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:31.935 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:01:31.935 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:31.996 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:01:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.100+0000 7f69af9dd640 1 Processor -- start 2024-05-11T01:01:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.101+0000 7f69af9dd640 1 -- start start 2024-05-11T01:01:32.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.101+0000 7f69af9dd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69a80b5dd0 0x7f69a80d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.101+0000 7f69af9dd640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a8163250 con 0x7f69a8160480 2024-05-11T01:01:32.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.101+0000 7f69af9dd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a8163530 con 0x7f69a815f430 2024-05-11T01:01:32.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.101+0000 7f69af9dd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f69a8163810 con 0x7f69a80b5dd0 2024-05-11T01:01:32.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69acf78640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f69a8160480 0x7f69a80d2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51676/0 (socket says 172.21.15.52:51676) 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69acf78640 1 -- 172.21.15.52:0/560386369 learned_addr learned my addr 172.21.15.52:0/560386369 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903179957 0 0) 0x7f69a8163250 con 0x7f69a8160480 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69adf7a640 1 --2- 172.21.15.52:0/560386369 >> v2:172.21.15.119:3300/0 conn(0x7f69a80b5dd0 0x7f69a80d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6990003520 con 0x7f69a8160480 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1682771152 0 0) 0x7f6990003520 con 0x7f69a8160480 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 >> v2:172.21.15.119:3300/0 conn(0x7f69a80b5dd0 msgr2=0x7f69a80d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 --2- 172.21.15.52:0/560386369 >> v2:172.21.15.119:3300/0 conn(0x7f69a80b5dd0 0x7f69a80d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 >> v1:172.21.15.119:6789/0 conn(0x7f69a815f430 legacy=0x7f69a815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a8163af0 con 0x7f69a8160480 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f699c0016f0 con 0x7f69a8160480 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 >> v1:172.21.15.52:6789/0 conn(0x7f69a8160480 legacy=0x7f69a80d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6990003a50 0x7f6990003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6990004800 0x7f6990004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69900052e0 0x7f699000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8163810 con 0x7f6990004800 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f699000c1b0 con 0x7f69900052e0 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f699000c3b0 con 0x7f6990003a50 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69acf78640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6990004800 0x7f6990004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69ad779640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6990003a50 0x7f6990003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.102+0000 7f69adf7a640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69900052e0 0x7f699000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69acf78640 1 -- 172.21.15.52:0/560386369 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6990003a50 msgr2=0x7f6990003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69acf78640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6990003a50 0x7f6990003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69acf78640 1 -- 172.21.15.52:0/560386369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69900052e0 msgr2=0x7f699000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69acf78640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69900052e0 0x7f699000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69acf78640 1 -- 172.21.15.52:0/560386369 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6990003520 con 0x7f6990004800 2024-05-11T01:01:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69adf7a640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69900052e0 0x7f699000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69adf7a640 1 -- 172.21.15.52:0/560386369 reap_dead start 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69acf78640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6990004800 0x7f6990004c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f699c001eb0 tx=0x7f6990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3193ef0108114f71 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c007d10 con 0x7f6990004800 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.103+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699c005ad0 con 0x7f6990004800 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.104+0000 7f69a7fff640 1 -- 172.21.15.52:0/560386369 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c007070 con 0x7f6990004800 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.104+0000 7f69af9dd640 1 -- 172.21.15.52:0/560386369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6990004800 msgr2=0x7f6990004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.104+0000 7f69af9dd640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6990004800 0x7f6990004c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f699c001eb0 tx=0x7f6990005830 comp rx=0 tx=0).stop 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.104+0000 7f69af9dd640 1 -- 172.21.15.52:0/560386369 shutdown_connections 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.104+0000 7f69af9dd640 1 --2- 172.21.15.52:0/560386369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6990004800 0x7f6990004c80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.104+0000 7f69af9dd640 1 -- 172.21.15.52:0/560386369 >> 172.21.15.52:0/560386369 conn(0x7f69a808eb70 msgr2=0x7f69a809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.104+0000 7f69af9dd640 1 -- 172.21.15.52:0/560386369 shutdown_connections 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.104+0000 7f69af9dd640 1 -- 172.21.15.52:0/560386369 wait complete. 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.105+0000 7f69af9dd640 1 Processor -- start 2024-05-11T01:01:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.105+0000 7f69af9dd640 1 -- start start 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.106+0000 7f69af9dd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6990004800 0x7f69a8158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.106+0000 7f69af9dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69a80cb1e0 0x7f69a80cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.106+0000 7f69af9dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69a80b2e40 0x7f69a80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.106+0000 7f69af9dd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8163030 con 0x7f69a80b2e40 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.106+0000 7f69af9dd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8162eb0 con 0x7f69a80cb1e0 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.106+0000 7f69af9dd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f69a81608f0 con 0x7f6990004800 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.106+0000 7f69adf7a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69a80b2e40 0x7f69a80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.106+0000 7f69adf7a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69a80b2e40 0x7f69a80b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47126/0 (socket says 172.21.15.52:47126) 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.106+0000 7f69adf7a640 1 -- 172.21.15.52:0/742807216 learned_addr learned my addr 172.21.15.52:0/742807216 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69ad779640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6990004800 0x7f69a8158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69acf78640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69a80cb1e0 0x7f69a80cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69adf7a640 1 -- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6990004800 msgr2=0x7f69a8158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69adf7a640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6990004800 0x7f69a8158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69adf7a640 1 -- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69a80cb1e0 msgr2=0x7f69a80cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69adf7a640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69a80cb1e0 0x7f69a80cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69adf7a640 1 -- 172.21.15.52:0/742807216 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a8158a90 con 0x7f69a80b2e40 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69adf7a640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69a80b2e40 0x7f69a80b32c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69a808e750 tx=0x7f6998010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69a5ffb640 1 -- 172.21.15.52:0/742807216 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998010e00 con 0x7f69a80b2e40 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69a5ffb640 1 -- 172.21.15.52:0/742807216 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6998003990 con 0x7f69a80b2e40 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a80cc340 con 0x7f69a80b2e40 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69a5ffb640 1 -- 172.21.15.52:0/742807216 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998003d80 con 0x7f69a80b2e40 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.107+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a80cc900 con 0x7f69a80b2e40 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.108+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a8069780 con 0x7f69a80b2e40 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.109+0000 7f69a5ffb640 1 -- 172.21.15.52:0/742807216 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f699801e020 con 0x7f69a80b2e40 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.110+0000 7f69a5ffb640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f69940762d0 0x7f6994078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.110+0000 7f69a5ffb640 1 -- 172.21.15.52:0/742807216 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6998097c90 con 0x7f69a80b2e40 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.110+0000 7f69ad779640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f69940762d0 0x7f6994078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.110+0000 7f69ad779640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f69940762d0 0x7f6994078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69a00036b0 tx=0x7f69a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:32.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.114+0000 7f69a5ffb640 1 -- 172.21.15.52:0/742807216 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6998061680 con 0x7f69a80b2e40 2024-05-11T01:01:32.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.318+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69a80b4ba0 con 0x7f69a80b2e40 2024-05-11T01:01:32.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.318+0000 7f69a5ffb640 1 -- 172.21.15.52:0/742807216 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f6998060f80 con 0x7f69a80b2e40 2024-05-11T01:01:32.318 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:01:32.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f69940762d0 msgr2=0x7f6994078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f69940762d0 0x7f6994078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69a00036b0 tx=0x7f69a0009040 comp rx=0 tx=0).stop 2024-05-11T01:01:32.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69a80b2e40 msgr2=0x7f69a80b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69a80b2e40 0x7f69a80b32c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69a808e750 tx=0x7f6998010590 comp rx=0 tx=0).stop 2024-05-11T01:01:32.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 shutdown_connections 2024-05-11T01:01:32.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f69940762d0 0x7f6994078790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69a80b2e40 0x7f69a80b32c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69a80cb1e0 0x7f69a80cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 --2- 172.21.15.52:0/742807216 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6990004800 0x7f69a8158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 >> 172.21.15.52:0/742807216 conn(0x7f69a808eb70 msgr2=0x7f69a806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:32.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 shutdown_connections 2024-05-11T01:01:32.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.320+0000 7f69af9dd640 1 -- 172.21.15.52:0/742807216 wait complete. 2024-05-11T01:01:32.331 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:01:32.331 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:01:32.351 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:01:32.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.455+0000 7fb228de4640 1 Processor -- start 2024-05-11T01:01:32.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.455+0000 7fb228de4640 1 -- start start 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.456+0000 7fb228de4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb22415f580 0x7fb2240cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.456+0000 7fb228de4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb2240d1500 0x7fb2240d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.456+0000 7fb228de4640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2240d5be0 con 0x7fb2240d36f0 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.456+0000 7fb228de4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb2240d0460 con 0x7fb2240d1500 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.456+0000 7fb228de4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb2240d0740 con 0x7fb22415f580 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb221d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb2240d1500 0x7fb2240d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb222d9d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb2240d36f0 0x7fb2240d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51680/0 (socket says 172.21.15.52:51680) 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb222d9d640 1 -- 172.21.15.52:0/291546607 learned_addr learned my addr 172.21.15.52:0/291546607 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb221d9b640 1 -- 172.21.15.52:0/291546607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb22415f580 msgr2=0x7fb2240cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb22259c640 1 --2- 172.21.15.52:0/291546607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb22415f580 0x7fb2240cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb221d9b640 1 --2- 172.21.15.52:0/291546607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb22415f580 0x7fb2240cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb22159a640 1 -- 172.21.15.52:0/291546607 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2718345156 0 0) 0x7fb2240d5be0 con 0x7fb2240d36f0 2024-05-11T01:01:32.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb221d9b640 1 -- 172.21.15.52:0/291546607 >> v1:172.21.15.52:6789/0 conn(0x7fb2240d36f0 legacy=0x7fb2240d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb221d9b640 1 -- 172.21.15.52:0/291546607 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2240d0a20 con 0x7fb2240d1500 2024-05-11T01:01:32.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.457+0000 7fb22259c640 1 --2- 172.21.15.52:0/291546607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb22415f580 0x7fb2240cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:32.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb221d9b640 1 --2- 172.21.15.52:0/291546607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2240d1500 0x7fb2240d56a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2180097a0 tx=0x7fb21802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c41186f1a53945f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:32.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb22159a640 1 -- 172.21.15.52:0/291546607 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218037070 con 0x7fb2240d1500 2024-05-11T01:01:32.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb22159a640 1 -- 172.21.15.52:0/291546607 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb218032440 con 0x7fb2240d1500 2024-05-11T01:01:32.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb22159a640 1 -- 172.21.15.52:0/291546607 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218032810 con 0x7fb2240d1500 2024-05-11T01:01:32.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb228de4640 1 -- 172.21.15.52:0/291546607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2240d1500 msgr2=0x7fb2240d56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb228de4640 1 --2- 172.21.15.52:0/291546607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2240d1500 0x7fb2240d56a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2180097a0 tx=0x7fb21802a180 comp rx=0 tx=0).stop 2024-05-11T01:01:32.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb228de4640 1 -- 172.21.15.52:0/291546607 shutdown_connections 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb228de4640 1 --2- 172.21.15.52:0/291546607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2240d1500 0x7fb2240d56a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb228de4640 1 --2- 172.21.15.52:0/291546607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb22415f580 0x7fb2240cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb228de4640 1 -- 172.21.15.52:0/291546607 >> 172.21.15.52:0/291546607 conn(0x7fb22408eb80 msgr2=0x7fb22409e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.458+0000 7fb228de4640 1 -- 172.21.15.52:0/291546607 shutdown_connections 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.459+0000 7fb228de4640 1 -- 172.21.15.52:0/291546607 wait complete. 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.460+0000 7fb228de4640 1 Processor -- start 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.460+0000 7fb228de4640 1 -- start start 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb228de4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2240d1500 0x7fb2241f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb228de4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2240d36f0 0x7fb2241f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb228de4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22415f580 0x7fb2240b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb228de4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb22415f960 con 0x7fb22415f580 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb228de4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2240d0fe0 con 0x7fb2240d36f0 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb228de4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb2240d3ba0 con 0x7fb2240d1500 2024-05-11T01:01:32.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb222d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22415f580 0x7fb2240b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb222d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22415f580 0x7fb2240b7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47144/0 (socket says 172.21.15.52:47144) 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb222d9d640 1 -- 172.21.15.52:0/2874684371 learned_addr learned my addr 172.21.15.52:0/2874684371 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb22259c640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2240d1500 0x7fb2241f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb221d9b640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2240d36f0 0x7fb2241f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb222d9d640 1 -- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2240d1500 msgr2=0x7fb2241f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb222d9d640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2240d1500 0x7fb2241f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb222d9d640 1 -- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2240d36f0 msgr2=0x7fb2241f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb222d9d640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2240d36f0 0x7fb2241f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb222d9d640 1 -- 172.21.15.52:0/2874684371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2240b8690 con 0x7fb22415f580 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb222d9d640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22415f580 0x7fb2240b7d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00c7d0 tx=0x7fb21c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb212ffd640 1 -- 172.21.15.52:0/2874684371 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c018070 con 0x7fb22415f580 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb212ffd640 1 -- 172.21.15.52:0/2874684371 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21c00a5e0 con 0x7fb22415f580 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.461+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2240b8970 con 0x7fb22415f580 2024-05-11T01:01:32.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.462+0000 7fb212ffd640 1 -- 172.21.15.52:0/2874684371 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00aa10 con 0x7fb22415f580 2024-05-11T01:01:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.462+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2240b9910 con 0x7fb22415f580 2024-05-11T01:01:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.462+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb224069780 con 0x7fb22415f580 2024-05-11T01:01:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.463+0000 7fb212ffd640 1 -- 172.21.15.52:0/2874684371 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb21c01b070 con 0x7fb22415f580 2024-05-11T01:01:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.464+0000 7fb212ffd640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2040762d0 0x7fb204078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.464+0000 7fb212ffd640 1 -- 172.21.15.52:0/2874684371 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb21c096b50 con 0x7fb22415f580 2024-05-11T01:01:32.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.464+0000 7fb22259c640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2040762d0 0x7fb204078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.464+0000 7fb22259c640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2040762d0 0x7fb204078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb2241f82e0 tx=0x7fb214008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:32.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.468+0000 7fb212ffd640 1 -- 172.21.15.52:0/2874684371 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb21c05fe60 con 0x7fb22415f580 2024-05-11T01:01:32.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.675+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb2240767b0 con 0x7fb22415f580 2024-05-11T01:01:32.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.675+0000 7fb212ffd640 1 -- 172.21.15.52:0/2874684371 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fb21c064160 con 0x7fb22415f580 2024-05-11T01:01:32.675 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:01:32.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2040762d0 msgr2=0x7fb204078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2040762d0 0x7fb204078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb2241f82e0 tx=0x7fb214008040 comp rx=0 tx=0).stop 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22415f580 msgr2=0x7fb2240b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22415f580 0x7fb2240b7d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00c7d0 tx=0x7fb21c00cc90 comp rx=0 tx=0).stop 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 shutdown_connections 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb2040762d0 0x7fb204078790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22415f580 0x7fb2240b7d90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2240d36f0 0x7fb2241f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 --2- 172.21.15.52:0/2874684371 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2240d1500 0x7fb2241f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 >> 172.21.15.52:0/2874684371 conn(0x7fb22408eb80 msgr2=0x7fb2240cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 shutdown_connections 2024-05-11T01:01:32.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.677+0000 7fb228de4640 1 -- 172.21.15.52:0/2874684371 wait complete. 2024-05-11T01:01:32.689 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi119.front.sepia.ceph.com... 2024-05-11T01:01:32.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:32.689 DEBUG:teuthology.orchestra.run.smithi119:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-11T01:01:32.719 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:01:32.720 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:01:32.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:32.720 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-11T01:01:32.778 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-11T01:01:32.778 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:32.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.882+0000 7fea622cc640 1 Processor -- start 2024-05-11T01:01:32.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.882+0000 7fea622cc640 1 -- start start 2024-05-11T01:01:32.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.883+0000 7fea622cc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c15f430 0x7fea5c15fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.883+0000 7fea622cc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fea5c1602b0 0x7fea5c1606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.883+0000 7fea622cc640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea5c163250 con 0x7fea5c0d1b50 2024-05-11T01:01:32.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.883+0000 7fea622cc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fea5c163530 con 0x7fea5c1602b0 2024-05-11T01:01:32.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.883+0000 7fea622cc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c163810 con 0x7fea5c15f430 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.883+0000 7fea60869640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fea5c0d1b50 0x7fea5c0d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51688/0 (socket says 172.21.15.52:51688) 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.883+0000 7fea60869640 1 -- 172.21.15.52:0/83875227 learned_addr learned my addr 172.21.15.52:0/83875227 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.883+0000 7fea5bfff640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c15f430 0x7fea5c15fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.883+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/83875227 >> v2:172.21.15.119:3300/0 conn(0x7fea5c1602b0 0x7fea5c1606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752867540 0 0) 0x7fea5c163250 con 0x7fea5c0d1b50 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea44003520 con 0x7fea5c0d1b50 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2921552249 0 0) 0x7fea44003520 con 0x7fea5c0d1b50 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c15f430 msgr2=0x7fea5c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c15f430 0x7fea5c15fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 >> v2:172.21.15.119:3300/0 conn(0x7fea5c1602b0 msgr2=0x7fea5c1606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 --2- 172.21.15.52:0/83875227 >> v2:172.21.15.119:3300/0 conn(0x7fea5c1602b0 0x7fea5c1606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c163af0 con 0x7fea5c0d1b50 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fea54001a70 con 0x7fea5c0d1b50 2024-05-11T01:01:32.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 >> v1:172.21.15.52:6789/0 conn(0x7fea5c0d1b50 legacy=0x7fea5c0d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea44003a00 0x7fea44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c15f430 0x7fea44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c163530 con 0x7fea44004d40 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c163810 con 0x7fea44003a00 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fea4400b9c0 con 0x7fea5c15f430 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/83875227 >> v2:172.21.15.119:3300/0 conn(0x7fea5c1602b0 0x7fea5c1606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea60869640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5bfff640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea44003a00 0x7fea44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c15f430 0x7fea44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea60869640 1 -- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c15f430 msgr2=0x7fea44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea60869640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c15f430 0x7fea44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea60869640 1 -- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea44003a00 msgr2=0x7fea44003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:32.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea60869640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea44003a00 0x7fea44003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.884+0000 7fea60869640 1 -- 172.21.15.52:0/83875227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea44003520 con 0x7fea44004d40 2024-05-11T01:01:32.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.885+0000 7fea60869640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea440051c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea5400a920 tx=0x7fea44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57569a83901b54b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:32.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.885+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5400a4c0 con 0x7fea44004d40 2024-05-11T01:01:32.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.885+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5402f070 con 0x7fea44004d40 2024-05-11T01:01:32.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.885+0000 7fea5affd640 1 -- 172.21.15.52:0/83875227 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5402bc70 con 0x7fea44004d40 2024-05-11T01:01:32.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.886+0000 7fea622cc640 1 -- 172.21.15.52:0/83875227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 msgr2=0x7fea440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.886+0000 7fea622cc640 1 --2- 172.21.15.52:0/83875227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea440051c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea5400a920 tx=0x7fea44005830 comp rx=0 tx=0).stop 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.886+0000 7fea60869640 1 -- 172.21.15.52:0/83875227 reap_dead start 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.886+0000 7fea622cc640 1 -- 172.21.15.52:0/83875227 shutdown_connections 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.886+0000 7fea622cc640 1 -- 172.21.15.52:0/83875227 >> 172.21.15.52:0/83875227 conn(0x7fea5c08eb70 msgr2=0x7fea5c09e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.886+0000 7fea622cc640 1 -- 172.21.15.52:0/83875227 shutdown_connections 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.886+0000 7fea622cc640 1 -- 172.21.15.52:0/83875227 wait complete. 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.887+0000 7fea622cc640 1 Processor -- start 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.887+0000 7fea622cc640 1 -- start start 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea622cc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea5c153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea622cc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea5c0cc210 0x7fea5c1f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea622cc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c150460 0x7fea5c0ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea622cc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c163030 con 0x7fea44004d40 2024-05-11T01:01:32.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea622cc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c162eb0 con 0x7fea5c0cc210 2024-05-11T01:01:32.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea622cc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c0d2710 con 0x7fea5c150460 2024-05-11T01:01:32.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea5c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea5c153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47166/0 (socket says 172.21.15.52:47166) 2024-05-11T01:01:32.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea5bfff640 1 -- 172.21.15.52:0/211843602 learned_addr learned my addr 172.21.15.52:0/211843602 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:32.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea60869640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c150460 0x7fea5c0ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea5bfff640 1 -- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c150460 msgr2=0x7fea5c0ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea5bfff640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c150460 0x7fea5c0ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea5bfff640 1 -- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea5c0cc210 msgr2=0x7fea5c1f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea5bfff640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea5c0cc210 0x7fea5c1f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea5bfff640 1 -- 172.21.15.52:0/211843602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c1f5420 con 0x7fea44004d40 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.888+0000 7fea5bfff640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea5c153970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0118e0 tx=0x7fea4c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.889+0000 7fea58ff9640 1 -- 172.21.15.52:0/211843602 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c00faf0 con 0x7fea44004d40 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.889+0000 7fea58ff9640 1 -- 172.21.15.52:0/211843602 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4c00fc90 con 0x7fea44004d40 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.889+0000 7fea58ff9640 1 -- 172.21.15.52:0/211843602 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c014440 con 0x7fea44004d40 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.889+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c1f56c0 con 0x7fea44004d40 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.889+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c1f5c80 con 0x7fea44004d40 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.890+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c069780 con 0x7fea44004d40 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.890+0000 7fea58ff9640 1 -- 172.21.15.52:0/211843602 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fea4c012040 con 0x7fea44004d40 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.891+0000 7fea58ff9640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fea480762d0 0x7fea48078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.891+0000 7fea58ff9640 1 -- 172.21.15.52:0/211843602 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea4c097ce0 con 0x7fea44004d40 2024-05-11T01:01:32.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.891+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fea480762d0 0x7fea48078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:32.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.892+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fea480762d0 0x7fea48078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0cc630 tx=0x7fea50009780 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:32.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:32.895+0000 7fea58ff9640 1 -- 172.21.15.52:0/211843602 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fea4c019030 con 0x7fea44004d40 2024-05-11T01:01:33.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.128+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5c0b4ba0 con 0x7fea44004d40 2024-05-11T01:01:33.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.128+0000 7fea58ff9640 1 -- 172.21.15.52:0/211843602 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fea4c061640 con 0x7fea44004d40 2024-05-11T01:01:33.128 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:33.129 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2245862597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2245862597},{"type":"v1","addr":"172.21.15.119:6837","nonce":2245862597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4286578298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4286578298},{"type":"v1","addr":"172.21.15.52:6835","nonce":4286578298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:01.052672+0000","modified":"2024-05-11T01:01:04.065799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/1484114523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1484114523},{"type":"v1","addr":"172.21.15.119:6836","nonce":1484114523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-11T01:01:33.129 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-05-11T01:01:33.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fea480762d0 msgr2=0x7fea48078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fea480762d0 0x7fea48078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0cc630 tx=0x7fea50009780 comp rx=0 tx=0).stop 2024-05-11T01:01:33.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 msgr2=0x7fea5c153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea5c153970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0118e0 tx=0x7fea4c011da0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 shutdown_connections 2024-05-11T01:01:33.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fea480762d0 0x7fea48078790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea5c150460 0x7fea5c0ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea5c0cc210 0x7fea5c1f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 --2- 172.21.15.52:0/211843602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea44004d40 0x7fea5c153970 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 >> 172.21.15.52:0/211843602 conn(0x7fea5c08eb70 msgr2=0x7fea5c0d3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:33.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 shutdown_connections 2024-05-11T01:01:33.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.130+0000 7fea622cc640 1 -- 172.21.15.52:0/211843602 wait complete. 2024-05-11T01:01:33.142 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:33.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.245+0000 7f058b351640 1 Processor -- start 2024-05-11T01:01:33.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.245+0000 7f058b351640 1 -- start start 2024-05-11T01:01:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f058b351640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f058415f430 0x7f058415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f058b351640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f05841606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f058b351640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0584163250 con 0x7f05840d1b50 2024-05-11T01:01:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f058b351640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0584163530 con 0x7f058415f430 2024-05-11T01:01:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f058b351640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f0584163810 con 0x7f05841602b0 2024-05-11T01:01:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f05888ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f05841606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f05890ed640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f058415f430 0x7f058415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f05898ee640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f05840d1b50 0x7f05840d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:42330/0 (socket says 172.21.15.52:42330) 2024-05-11T01:01:33.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f05888ec640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f05841606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:49160/0 (socket says 172.21.15.52:49160) 2024-05-11T01:01:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.246+0000 7f05898ee640 1 -- 172.21.15.52:0/3358813357 learned_addr learned my addr 172.21.15.52:0/3358813357 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.247+0000 7f05890ed640 1 -- 172.21.15.52:0/3358813357 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 msgr2=0x7f05841606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.247+0000 7f05890ed640 1 --2- 172.21.15.52:0/3358813357 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f05841606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.247+0000 7f05890ed640 1 -- 172.21.15.52:0/3358813357 >> v1:172.21.15.119:6789/0 conn(0x7f05840d1b50 legacy=0x7f05840d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.247+0000 7f0583fff640 1 -- 172.21.15.52:0/3358813357 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472433010 0 0) 0x7f0584163250 con 0x7f05840d1b50 2024-05-11T01:01:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.247+0000 7f05890ed640 1 -- 172.21.15.52:0/3358813357 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0584163af0 con 0x7f058415f430 2024-05-11T01:01:33.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.247+0000 7f05888ec640 1 --2- 172.21.15.52:0/3358813357 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f05841606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:33.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f05890ed640 1 --2- 172.21.15.52:0/3358813357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 0x7f058415fd70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f057c0097a0 tx=0x7f057c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8c1b9a68a03aeed server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:33.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f0583fff640 1 -- 172.21.15.52:0/3358813357 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c037070 con 0x7f058415f430 2024-05-11T01:01:33.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f0583fff640 1 -- 172.21.15.52:0/3358813357 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c032440 con 0x7f058415f430 2024-05-11T01:01:33.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f0583fff640 1 -- 172.21.15.52:0/3358813357 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c032810 con 0x7f058415f430 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f058b351640 1 -- 172.21.15.52:0/3358813357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 msgr2=0x7f058415fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f058b351640 1 --2- 172.21.15.52:0/3358813357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 0x7f058415fd70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f057c0097a0 tx=0x7f057c02a180 comp rx=0 tx=0).stop 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f058b351640 1 -- 172.21.15.52:0/3358813357 shutdown_connections 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f058b351640 1 --2- 172.21.15.52:0/3358813357 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f05841606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f058b351640 1 --2- 172.21.15.52:0/3358813357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 0x7f058415fd70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f058b351640 1 -- 172.21.15.52:0/3358813357 >> 172.21.15.52:0/3358813357 conn(0x7f058408eb70 msgr2=0x7f058409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f058b351640 1 -- 172.21.15.52:0/3358813357 shutdown_connections 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.248+0000 7f058b351640 1 -- 172.21.15.52:0/3358813357 wait complete. 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.249+0000 7f058b351640 1 Processor -- start 2024-05-11T01:01:33.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f058b351640 1 -- start start 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f058b351640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840d1b50 0x7f05840b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f058b351640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 0x7f058415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f058b351640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f0584161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f058b351640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0584163030 con 0x7f05840d1b50 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f058b351640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0584162eb0 con 0x7f058415f430 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f058b351640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f05840d2710 con 0x7f05841602b0 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f05890ed640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840d1b50 0x7f05840b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f05890ed640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840d1b50 0x7f05840b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47170/0 (socket says 172.21.15.52:47170) 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.250+0000 7f05890ed640 1 -- 172.21.15.52:0/2642460704 learned_addr learned my addr 172.21.15.52:0/2642460704 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f05888ec640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 0x7f058415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f05898ee640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f0584161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f05890ed640 1 -- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 msgr2=0x7f0584161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f05890ed640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f0584161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f05890ed640 1 -- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 msgr2=0x7f058415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f05890ed640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 0x7f058415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f05890ed640 1 -- 172.21.15.52:0/2642460704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05840c99d0 con 0x7f05840d1b50 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f05888ec640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 0x7f058415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f05890ed640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840d1b50 0x7f05840b2800 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f057c0097a0 tx=0x7f057c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f0581ffb640 1 -- 172.21.15.52:0/2642460704 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c037050 con 0x7f05840d1b50 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f0581ffb640 1 -- 172.21.15.52:0/2642460704 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c038040 con 0x7f05840d1b50 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05840cdf60 con 0x7f05840d1b50 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f0581ffb640 1 -- 172.21.15.52:0/2642460704 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c004490 con 0x7f05840d1b50 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.251+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05840ce520 con 0x7f05840d1b50 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.252+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0584069780 con 0x7f05840d1b50 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.253+0000 7f0581ffb640 1 -- 172.21.15.52:0/2642460704 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f057c03c6d0 con 0x7f05840d1b50 2024-05-11T01:01:33.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.254+0000 7f0581ffb640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f05700762d0 0x7f0570078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.254+0000 7f0581ffb640 1 -- 172.21.15.52:0/2642460704 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f057c0b80c0 con 0x7f05840d1b50 2024-05-11T01:01:33.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.254+0000 7f05888ec640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f05700762d0 0x7f0570078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.254+0000 7f05888ec640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f05700762d0 0x7f0570078790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f058415de50 tx=0x7f0578008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:33.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.258+0000 7f0581ffb640 1 -- 172.21.15.52:0/2642460704 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f05840c99d0 con 0x7f05840d1b50 2024-05-11T01:01:33.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.487+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f058415df20 con 0x7f05840d1b50 2024-05-11T01:01:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.488+0000 7f0581ffb640 1 -- 172.21.15.52:0/2642460704 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f057c081990 con 0x7f05840d1b50 2024-05-11T01:01:33.488 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:33.488 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2245862597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2245862597},{"type":"v1","addr":"172.21.15.119:6837","nonce":2245862597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4286578298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4286578298},{"type":"v1","addr":"172.21.15.52:6835","nonce":4286578298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:01.052672+0000","modified":"2024-05-11T01:01:04.065799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/1484114523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1484114523},{"type":"v1","addr":"172.21.15.119:6836","nonce":1484114523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-11T01:01:33.488 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-05-11T01:01:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f05700762d0 msgr2=0x7f0570078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f05700762d0 0x7f0570078790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f058415de50 tx=0x7f0578008040 comp rx=0 tx=0).stop 2024-05-11T01:01:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840d1b50 msgr2=0x7f05840b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840d1b50 0x7f05840b2800 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f057c0097a0 tx=0x7f057c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 shutdown_connections 2024-05-11T01:01:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f05700762d0 0x7f0570078790 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f05841602b0 0x7f0584161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f058415f430 0x7f058415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 --2- 172.21.15.52:0/2642460704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840d1b50 0x7f05840b2800 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 >> 172.21.15.52:0/2642460704 conn(0x7f058408eb70 msgr2=0x7f058409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 shutdown_connections 2024-05-11T01:01:33.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.490+0000 7f058b351640 1 -- 172.21.15.52:0/2642460704 wait complete. 2024-05-11T01:01:33.501 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:33.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.605+0000 7fb70e6e3640 1 Processor -- start 2024-05-11T01:01:33.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.605+0000 7fb70e6e3640 1 -- start start 2024-05-11T01:01:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.605+0000 7fb70e6e3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb7080cf170 0x7fb708073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.605+0000 7fb70e6e3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb708160bd0 0x7fb708161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.605+0000 7fb70e6e3640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb70815d4a0 con 0x7fb70806c480 2024-05-11T01:01:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.605+0000 7fb70e6e3640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb7080cece0 con 0x7fb708160bd0 2024-05-11T01:01:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.605+0000 7fb70e6e3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb7080d1680 con 0x7fb7080cf170 2024-05-11T01:01:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb70cc80640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb708160bd0 0x7fb708161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb707fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb70806c480 0x7fb70806c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51700/0 (socket says 172.21.15.52:51700) 2024-05-11T01:01:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb707fff640 1 -- 172.21.15.52:0/4015201428 learned_addr learned my addr 172.21.15.52:0/4015201428 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb6fffff640 1 --2- 172.21.15.52:0/4015201428 >> v2:172.21.15.119:3300/0 conn(0x7fb7080cf170 0x7fb708073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb70cc80640 1 -- 172.21.15.52:0/4015201428 >> v2:172.21.15.119:3300/0 conn(0x7fb7080cf170 msgr2=0x7fb708073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb70cc80640 1 --2- 172.21.15.52:0/4015201428 >> v2:172.21.15.119:3300/0 conn(0x7fb7080cf170 0x7fb708073fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb70cc80640 1 -- 172.21.15.52:0/4015201428 >> v1:172.21.15.52:6789/0 conn(0x7fb70806c480 legacy=0x7fb70806c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb70cc80640 1 -- 172.21.15.52:0/4015201428 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7080d1960 con 0x7fb708160bd0 2024-05-11T01:01:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb7077fe640 1 -- 172.21.15.52:0/4015201428 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 612735621 0 0) 0x7fb70815d4a0 con 0x7fb70806c480 2024-05-11T01:01:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.606+0000 7fb6fffff640 1 --2- 172.21.15.52:0/4015201428 >> v2:172.21.15.119:3300/0 conn(0x7fb7080cf170 0x7fb708073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb70cc80640 1 --2- 172.21.15.52:0/4015201428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb708160bd0 0x7fb708161060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb7080a00b0 tx=0x7fb70002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb7a7c367f2475e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb7077fe640 1 -- 172.21.15.52:0/4015201428 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700007740 con 0x7fb708160bd0 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb7077fe640 1 -- 172.21.15.52:0/4015201428 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7000078e0 con 0x7fb708160bd0 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb7077fe640 1 -- 172.21.15.52:0/4015201428 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700007cb0 con 0x7fb708160bd0 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb70e6e3640 1 -- 172.21.15.52:0/4015201428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb708160bd0 msgr2=0x7fb708161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb70e6e3640 1 --2- 172.21.15.52:0/4015201428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb708160bd0 0x7fb708161060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb7080a00b0 tx=0x7fb70002a680 comp rx=0 tx=0).stop 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb70e6e3640 1 -- 172.21.15.52:0/4015201428 shutdown_connections 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb70e6e3640 1 --2- 172.21.15.52:0/4015201428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb708160bd0 0x7fb708161060 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb70e6e3640 1 --2- 172.21.15.52:0/4015201428 >> v2:172.21.15.119:3300/0 conn(0x7fb7080cf170 0x7fb708073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.607+0000 7fb70e6e3640 1 -- 172.21.15.52:0/4015201428 >> 172.21.15.52:0/4015201428 conn(0x7fb7080b3a10 msgr2=0x7fb7080b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.608+0000 7fb70e6e3640 1 -- 172.21.15.52:0/4015201428 shutdown_connections 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.608+0000 7fb70e6e3640 1 -- 172.21.15.52:0/4015201428 wait complete. 2024-05-11T01:01:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.609+0000 7fb70e6e3640 1 Processor -- start 2024-05-11T01:01:33.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.609+0000 7fb70e6e3640 1 -- start start 2024-05-11T01:01:33.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.609+0000 7fb70e6e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb70806c480 0x7fb7080b6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.609+0000 7fb70e6e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7080cf170 0x7fb708152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.609+0000 7fb70e6e3640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb708160bd0 0x7fb7080b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.609+0000 7fb70e6e3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7080744f0 con 0x7fb70806c480 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.609+0000 7fb70e6e3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb7080cf5d0 con 0x7fb7080cf170 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.609+0000 7fb70e6e3640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb7080d1330 con 0x7fb708160bd0 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb707fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb70806c480 0x7fb7080b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb707fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb70806c480 0x7fb7080b6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47198/0 (socket says 172.21.15.52:47198) 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb707fff640 1 -- 172.21.15.52:0/3563698055 learned_addr learned my addr 172.21.15.52:0/3563698055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb6fffff640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7080cf170 0x7fb708152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb70cc80640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb708160bd0 0x7fb7080b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb707fff640 1 -- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb708160bd0 msgr2=0x7fb7080b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb707fff640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb708160bd0 0x7fb7080b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb707fff640 1 -- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7080cf170 msgr2=0x7fb708152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb707fff640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7080cf170 0x7fb708152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb707fff640 1 -- 172.21.15.52:0/3563698055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7080b5020 con 0x7fb70806c480 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb6fffff640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7080cf170 0x7fb708152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb707fff640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb70806c480 0x7fb7080b6560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800c490 tx=0x7fb6f800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb7057fa640 1 -- 172.21.15.52:0/3563698055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8018070 con 0x7fb70806c480 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb7057fa640 1 -- 172.21.15.52:0/3563698055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f8007500 con 0x7fb70806c480 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70815e7c0 con 0x7fb70806c480 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb7057fa640 1 -- 172.21.15.52:0/3563698055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8004a20 con 0x7fb70806c480 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.610+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70815ed80 con 0x7fb70806c480 2024-05-11T01:01:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.611+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb708069780 con 0x7fb70806c480 2024-05-11T01:01:33.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.612+0000 7fb7057fa640 1 -- 172.21.15.52:0/3563698055 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb6f801b070 con 0x7fb70806c480 2024-05-11T01:01:33.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.612+0000 7fb7057fa640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6e80762d0 0x7fb6e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.613+0000 7fb7057fa640 1 -- 172.21.15.52:0/3563698055 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb6f8096820 con 0x7fb70806c480 2024-05-11T01:01:33.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.613+0000 7fb6fffff640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6e80762d0 0x7fb6e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.613+0000 7fb6fffff640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6e80762d0 0x7fb6e8078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb708155d50 tx=0x7fb6f0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:33.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.617+0000 7fb7057fa640 1 -- 172.21.15.52:0/3563698055 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb6f8060210 con 0x7fb70806c480 2024-05-11T01:01:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.813+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7080b1660 con 0x7fb70806c480 2024-05-11T01:01:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.814+0000 7fb7057fa640 1 -- 172.21.15.52:0/3563698055 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fb6f805fb10 con 0x7fb70806c480 2024-05-11T01:01:33.814 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:33.815 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":20,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:02.673511+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:00:59.086890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:00.008780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6e80762d0 msgr2=0x7fb6e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6e80762d0 0x7fb6e8078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb708155d50 tx=0x7fb6f0008040 comp rx=0 tx=0).stop 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb70806c480 msgr2=0x7fb7080b6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb70806c480 0x7fb7080b6560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800c490 tx=0x7fb6f800c950 comp rx=0 tx=0).stop 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 shutdown_connections 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb6e80762d0 0x7fb6e8078790 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb708160bd0 0x7fb7080b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7080cf170 0x7fb708152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 --2- 172.21.15.52:0/3563698055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb70806c480 0x7fb7080b6560 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 >> 172.21.15.52:0/3563698055 conn(0x7fb7080b3a10 msgr2=0x7fb70806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 shutdown_connections 2024-05-11T01:01:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.816+0000 7fb70e6e3640 1 -- 172.21.15.52:0/3563698055 wait complete. 2024-05-11T01:01:33.829 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T01:01:33.829 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:33.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.933+0000 7fb96ff12640 1 Processor -- start 2024-05-11T01:01:33.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.933+0000 7fb96ff12640 1 -- start start 2024-05-11T01:01:33.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96ff12640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb968160480 0x7fb9680d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96ff12640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb9680d32c0 0x7fb9680d36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96ff12640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb968163250 con 0x7fb96815f430 2024-05-11T01:01:33.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96ff12640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb968163530 con 0x7fb9680d32c0 2024-05-11T01:01:33.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96ff12640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb968163810 con 0x7fb968160480 2024-05-11T01:01:33.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96e4af640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb9680d32c0 0x7fb9680d36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96dcae640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb96815f430 0x7fb96815fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51708/0 (socket says 172.21.15.52:51708) 2024-05-11T01:01:33.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96dcae640 1 -- 172.21.15.52:0/2663226398 learned_addr learned my addr 172.21.15.52:0/2663226398 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:33.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96d4ad640 1 --2- 172.21.15.52:0/2663226398 >> v2:172.21.15.119:3300/0 conn(0x7fb968160480 0x7fb9680d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96e4af640 1 -- 172.21.15.52:0/2663226398 >> v2:172.21.15.119:3300/0 conn(0x7fb968160480 msgr2=0x7fb9680d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:33.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96e4af640 1 --2- 172.21.15.52:0/2663226398 >> v2:172.21.15.119:3300/0 conn(0x7fb968160480 0x7fb9680d2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96e4af640 1 -- 172.21.15.52:0/2663226398 >> v1:172.21.15.52:6789/0 conn(0x7fb96815f430 legacy=0x7fb96815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96e4af640 1 -- 172.21.15.52:0/2663226398 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb968163af0 con 0x7fb9680d32c0 2024-05-11T01:01:33.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96ccac640 1 -- 172.21.15.52:0/2663226398 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 943287812 0 0) 0x7fb968163250 con 0x7fb96815f430 2024-05-11T01:01:33.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.934+0000 7fb96d4ad640 1 --2- 172.21.15.52:0/2663226398 >> v2:172.21.15.119:3300/0 conn(0x7fb968160480 0x7fb9680d2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:33.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.935+0000 7fb96e4af640 1 --2- 172.21.15.52:0/2663226398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 0x7fb9680d36c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9680b4840 tx=0x7fb96002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee2630ce6230956 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:33.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.935+0000 7fb96ccac640 1 -- 172.21.15.52:0/2663226398 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960037070 con 0x7fb9680d32c0 2024-05-11T01:01:33.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.935+0000 7fb96ccac640 1 -- 172.21.15.52:0/2663226398 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb960032440 con 0x7fb9680d32c0 2024-05-11T01:01:33.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.935+0000 7fb96ccac640 1 -- 172.21.15.52:0/2663226398 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96002ae60 con 0x7fb9680d32c0 2024-05-11T01:01:33.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.936+0000 7fb96ff12640 1 -- 172.21.15.52:0/2663226398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 msgr2=0x7fb9680d36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.936+0000 7fb96ff12640 1 --2- 172.21.15.52:0/2663226398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 0x7fb9680d36c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9680b4840 tx=0x7fb96002a110 comp rx=0 tx=0).stop 2024-05-11T01:01:33.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.936+0000 7fb96ff12640 1 -- 172.21.15.52:0/2663226398 shutdown_connections 2024-05-11T01:01:33.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.936+0000 7fb96ff12640 1 --2- 172.21.15.52:0/2663226398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 0x7fb9680d36c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.936+0000 7fb96ff12640 1 --2- 172.21.15.52:0/2663226398 >> v2:172.21.15.119:3300/0 conn(0x7fb968160480 0x7fb9680d2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.936+0000 7fb96ff12640 1 -- 172.21.15.52:0/2663226398 >> 172.21.15.52:0/2663226398 conn(0x7fb96808eb70 msgr2=0x7fb96809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:33.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.936+0000 7fb96ff12640 1 -- 172.21.15.52:0/2663226398 shutdown_connections 2024-05-11T01:01:33.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.936+0000 7fb96ff12640 1 -- 172.21.15.52:0/2663226398 wait complete. 2024-05-11T01:01:33.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.937+0000 7fb96ff12640 1 Processor -- start 2024-05-11T01:01:33.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.937+0000 7fb96ff12640 1 -- start start 2024-05-11T01:01:33.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96ff12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 0x7fb968156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96ff12640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb96815f430 0x7fb9680ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96ff12640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb968160480 0x7fb9680cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96ff12640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb968163030 con 0x7fb9680d32c0 2024-05-11T01:01:33.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96ff12640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb968162eb0 con 0x7fb968160480 2024-05-11T01:01:33.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96ff12640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb9681608f0 con 0x7fb96815f430 2024-05-11T01:01:33.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96dcae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 0x7fb968156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96dcae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 0x7fb968156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47220/0 (socket says 172.21.15.52:47220) 2024-05-11T01:01:33.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96dcae640 1 -- 172.21.15.52:0/71907261 learned_addr learned my addr 172.21.15.52:0/71907261 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:33.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96d4ad640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb96815f430 0x7fb9680ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96e4af640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb968160480 0x7fb9680cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96dcae640 1 -- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb96815f430 msgr2=0x7fb9680ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96dcae640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb96815f430 0x7fb9680ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96dcae640 1 -- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb968160480 msgr2=0x7fb9680cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:33.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96dcae640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb968160480 0x7fb9680cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:33.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96dcae640 1 -- 172.21.15.52:0/71907261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96815c150 con 0x7fb9680d32c0 2024-05-11T01:01:33.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96d4ad640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb96815f430 0x7fb9680ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:33.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96e4af640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb968160480 0x7fb9680cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb96dcae640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 0x7fb968156810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb96400c870 tx=0x7fb96400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.938+0000 7fb95e7fc640 1 -- 172.21.15.52:0/71907261 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964018070 con 0x7fb9680d32c0 2024-05-11T01:01:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.939+0000 7fb95e7fc640 1 -- 172.21.15.52:0/71907261 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96400a5e0 con 0x7fb9680d32c0 2024-05-11T01:01:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.939+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96815b500 con 0x7fb9680d32c0 2024-05-11T01:01:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.939+0000 7fb95e7fc640 1 -- 172.21.15.52:0/71907261 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96400a9d0 con 0x7fb9680d32c0 2024-05-11T01:01:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.939+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96815af40 con 0x7fb9680d32c0 2024-05-11T01:01:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.940+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb968069780 con 0x7fb9680d32c0 2024-05-11T01:01:33.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.940+0000 7fb95e7fc640 1 -- 172.21.15.52:0/71907261 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb96401b070 con 0x7fb9680d32c0 2024-05-11T01:01:33.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.941+0000 7fb95e7fc640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb94c076300 0x7fb94c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:33.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.941+0000 7fb95e7fc640 1 -- 172.21.15.52:0/71907261 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb964096c20 con 0x7fb9680d32c0 2024-05-11T01:01:33.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.941+0000 7fb96d4ad640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb94c076300 0x7fb94c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:33.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.942+0000 7fb96d4ad640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb94c076300 0x7fb94c0787c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb958006fd0 tx=0x7fb958008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:33.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:33.945+0000 7fb95e7fc640 1 -- 172.21.15.52:0/71907261 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb964060580 con 0x7fb9680d32c0 2024-05-11T01:01:34.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.178+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9680b4ba0 con 0x7fb9680d32c0 2024-05-11T01:01:34.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.179+0000 7fb95e7fc640 1 -- 172.21.15.52:0/71907261 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fb964063450 con 0x7fb9680d32c0 2024-05-11T01:01:34.179 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:34.180 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2245862597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2245862597},{"type":"v1","addr":"172.21.15.119:6837","nonce":2245862597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4286578298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4286578298},{"type":"v1","addr":"172.21.15.52:6835","nonce":4286578298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:01.052672+0000","modified":"2024-05-11T01:01:04.065799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/1484114523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1484114523},{"type":"v1","addr":"172.21.15.119:6836","nonce":1484114523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-11T01:01:34.180 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-05-11T01:01:34.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb94c076300 msgr2=0x7fb94c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb94c076300 0x7fb94c0787c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb958006fd0 tx=0x7fb958008040 comp rx=0 tx=0).stop 2024-05-11T01:01:34.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 msgr2=0x7fb968156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 0x7fb968156810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb96400c870 tx=0x7fb96400cd30 comp rx=0 tx=0).stop 2024-05-11T01:01:34.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 shutdown_connections 2024-05-11T01:01:34.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb94c076300 0x7fb94c0787c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb968160480 0x7fb9680cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb96815f430 0x7fb9680ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 --2- 172.21.15.52:0/71907261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9680d32c0 0x7fb968156810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 >> 172.21.15.52:0/71907261 conn(0x7fb96808eb70 msgr2=0x7fb9680d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:34.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 shutdown_connections 2024-05-11T01:01:34.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.181+0000 7fb96ff12640 1 -- 172.21.15.52:0/71907261 wait complete. 2024-05-11T01:01:34.192 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:34.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.295+0000 7fbb64c6b640 1 Processor -- start 2024-05-11T01:01:34.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.295+0000 7fbb64c6b640 1 -- start start 2024-05-11T01:01:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.296+0000 7fbb64c6b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 0x7fbb600d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.296+0000 7fbb64c6b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbb600b55b0 0x7fbb600ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.296+0000 7fbb64c6b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb600cd6e0 con 0x7fbb600b8a10 2024-05-11T01:01:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.296+0000 7fbb64c6b640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fbb600ce2f0 con 0x7fbb600b55b0 2024-05-11T01:01:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.296+0000 7fbb64c6b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbb600cd9c0 con 0x7fbb6009c130 2024-05-11T01:01:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5e59c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbb600b8a10 0x7fbb6009ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51724/0 (socket says 172.21.15.52:51724) 2024-05-11T01:01:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5e59c640 1 -- 172.21.15.52:0/283467817 learned_addr learned my addr 172.21.15.52:0/283467817 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5ed9d640 1 --2- 172.21.15.52:0/283467817 >> v2:172.21.15.52:3300/0 conn(0x7fbb600b55b0 0x7fbb600ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5dd9b640 1 --2- 172.21.15.52:0/283467817 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 0x7fbb600d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5d59a640 1 -- 172.21.15.52:0/283467817 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3795571804 0 0) 0x7fbb600cd6e0 con 0x7fbb600b8a10 2024-05-11T01:01:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5d59a640 1 -- 172.21.15.52:0/283467817 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb4c003520 con 0x7fbb600b8a10 2024-05-11T01:01:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5ed9d640 1 -- 172.21.15.52:0/283467817 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 msgr2=0x7fbb600d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5ed9d640 1 --2- 172.21.15.52:0/283467817 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 0x7fbb600d1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5ed9d640 1 -- 172.21.15.52:0/283467817 >> v1:172.21.15.52:6789/0 conn(0x7fbb600b8a10 legacy=0x7fbb6009ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.297+0000 7fbb5ed9d640 1 -- 172.21.15.52:0/283467817 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb600cdca0 con 0x7fbb600b55b0 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb5ed9d640 1 --2- 172.21.15.52:0/283467817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 0x7fbb600ce780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009d90 tx=0x7fbb5802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5892911a9a9fe29c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb5d59a640 1 -- 172.21.15.52:0/283467817 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58031600 con 0x7fbb600b55b0 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb5d59a640 1 -- 172.21.15.52:0/283467817 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb580317a0 con 0x7fbb600b55b0 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb5d59a640 1 -- 172.21.15.52:0/283467817 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58031b70 con 0x7fbb600b55b0 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb64c6b640 1 -- 172.21.15.52:0/283467817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 msgr2=0x7fbb600ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb64c6b640 1 --2- 172.21.15.52:0/283467817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 0x7fbb600ce780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009d90 tx=0x7fbb5802a5d0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb64c6b640 1 -- 172.21.15.52:0/283467817 shutdown_connections 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb64c6b640 1 --2- 172.21.15.52:0/283467817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 0x7fbb600ce780 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb64c6b640 1 --2- 172.21.15.52:0/283467817 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 0x7fbb600d1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb64c6b640 1 -- 172.21.15.52:0/283467817 >> 172.21.15.52:0/283467817 conn(0x7fbb600b4a90 msgr2=0x7fbb600b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.298+0000 7fbb64c6b640 1 -- 172.21.15.52:0/283467817 shutdown_connections 2024-05-11T01:01:34.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.299+0000 7fbb64c6b640 1 -- 172.21.15.52:0/283467817 wait complete. 2024-05-11T01:01:34.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.300+0000 7fbb64c6b640 1 Processor -- start 2024-05-11T01:01:34.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.300+0000 7fbb64c6b640 1 -- start start 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb64c6b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 0x7fbb601f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb64c6b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 0x7fbb60160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb64c6b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb600b8a10 0x7fbb601513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb64c6b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600b5db0 con 0x7fbb600b55b0 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb64c6b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600b8df0 con 0x7fbb600b8a10 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb64c6b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6006ee60 con 0x7fbb6009c130 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 0x7fbb60160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 0x7fbb60160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47228/0 (socket says 172.21.15.52:47228) 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5dd9b640 1 -- 172.21.15.52:0/3072353244 learned_addr learned my addr 172.21.15.52:0/3072353244 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5e59c640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 0x7fbb601f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5ed9d640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb600b8a10 0x7fbb601513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5dd9b640 1 -- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 msgr2=0x7fbb601f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5dd9b640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 0x7fbb601f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5dd9b640 1 -- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb600b8a10 msgr2=0x7fbb601513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5dd9b640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb600b8a10 0x7fbb601513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5dd9b640 1 -- 172.21.15.52:0/3072353244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb601605d0 con 0x7fbb600b55b0 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb5dd9b640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 0x7fbb60160090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e8a0 tx=0x7fbb4800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb56ffd640 1 -- 172.21.15.52:0/3072353244 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48017070 con 0x7fbb600b55b0 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb56ffd640 1 -- 172.21.15.52:0/3072353244 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4800c8e0 con 0x7fbb600b55b0 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb601f5380 con 0x7fbb600b55b0 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb56ffd640 1 -- 172.21.15.52:0/3072353244 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800ccd0 con 0x7fbb600b55b0 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.301+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb601f5940 con 0x7fbb600b55b0 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.302+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb60069780 con 0x7fbb600b55b0 2024-05-11T01:01:34.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.303+0000 7fbb56ffd640 1 -- 172.21.15.52:0/3072353244 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbb4801a070 con 0x7fbb600b55b0 2024-05-11T01:01:34.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.304+0000 7fbb56ffd640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbb400762d0 0x7fbb40078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.304+0000 7fbb56ffd640 1 -- 172.21.15.52:0/3072353244 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbb48097500 con 0x7fbb600b55b0 2024-05-11T01:01:34.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.304+0000 7fbb5e59c640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbb400762d0 0x7fbb40078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.304+0000 7fbb5e59c640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbb400762d0 0x7fbb40078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbb50007a60 tx=0x7fbb50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:34.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.308+0000 7fbb56ffd640 1 -- 172.21.15.52:0/3072353244 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbb601605d0 con 0x7fbb600b55b0 2024-05-11T01:01:34.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.537+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb60151920 con 0x7fbb600b55b0 2024-05-11T01:01:34.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.538+0000 7fbb56ffd640 1 -- 172.21.15.52:0/3072353244 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fbb48060dc0 con 0x7fbb600b55b0 2024-05-11T01:01:34.538 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:34.538 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2245862597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2245862597},{"type":"v1","addr":"172.21.15.119:6837","nonce":2245862597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/194782731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":194782731},{"type":"v1","addr":"172.21.15.52:6834","nonce":194782731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4286578298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4286578298},{"type":"v1","addr":"172.21.15.52:6835","nonce":4286578298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:01.052672+0000","modified":"2024-05-11T01:01:04.065799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/1484114523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1484114523},{"type":"v1","addr":"172.21.15.119:6836","nonce":1484114523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-11T01:01:34.538 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-05-11T01:01:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbb400762d0 msgr2=0x7fbb40078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbb400762d0 0x7fbb40078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbb50007a60 tx=0x7fbb50009040 comp rx=0 tx=0).stop 2024-05-11T01:01:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 msgr2=0x7fbb60160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 0x7fbb60160090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e8a0 tx=0x7fbb4800ed60 comp rx=0 tx=0).stop 2024-05-11T01:01:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 shutdown_connections 2024-05-11T01:01:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbb400762d0 0x7fbb40078790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb600b8a10 0x7fbb601513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb600b55b0 0x7fbb60160090 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 --2- 172.21.15.52:0/3072353244 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbb6009c130 0x7fbb601f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 >> 172.21.15.52:0/3072353244 conn(0x7fbb600b4a90 msgr2=0x7fbb60071a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 shutdown_connections 2024-05-11T01:01:34.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.540+0000 7fbb64c6b640 1 -- 172.21.15.52:0/3072353244 wait complete. 2024-05-11T01:01:34.551 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:34.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.655+0000 7f073af46640 1 Processor -- start 2024-05-11T01:01:34.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.655+0000 7f073af46640 1 -- start start 2024-05-11T01:01:34.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f073af46640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f073415f430 0x7f073415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f073af46640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 0x7f07340d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f073af46640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0734163250 con 0x7f07341602b0 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f073af46640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f0734163530 con 0x7f073415f430 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f073af46640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f0734163810 con 0x7f07340d3d90 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f0738ce2640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f073415f430 0x7f073415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f072bfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f07341602b0 0x7f07340d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51738/0 (socket says 172.21.15.52:51738) 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f0738ce2640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f073415f430 0x7f073415fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47234/0 (socket says 172.21.15.52:47234) 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f072bfff640 1 -- 172.21.15.52:0/762329182 learned_addr learned my addr 172.21.15.52:0/762329182 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f07394e3640 1 --2- 172.21.15.52:0/762329182 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 0x7f07340d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f0738ce2640 1 -- 172.21.15.52:0/762329182 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 msgr2=0x7f07340d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f0738ce2640 1 --2- 172.21.15.52:0/762329182 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 0x7f07340d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f0738ce2640 1 -- 172.21.15.52:0/762329182 >> v1:172.21.15.52:6789/0 conn(0x7f07341602b0 legacy=0x7f07340d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f0738ce2640 1 -- 172.21.15.52:0/762329182 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0734163af0 con 0x7f073415f430 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f072b7fe640 1 -- 172.21.15.52:0/762329182 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4135284877 0 0) 0x7f0734163250 con 0x7f07341602b0 2024-05-11T01:01:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.656+0000 7f07394e3640 1 --2- 172.21.15.52:0/762329182 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 0x7f07340d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.657+0000 7f0738ce2640 1 --2- 172.21.15.52:0/762329182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f073415f430 0x7f073415fd70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f07300097a0 tx=0x7f073002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22480418c2ea5f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.657+0000 7f072b7fe640 1 -- 172.21.15.52:0/762329182 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730037070 con 0x7f073415f430 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.657+0000 7f072b7fe640 1 -- 172.21.15.52:0/762329182 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0730032440 con 0x7f073415f430 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.657+0000 7f072b7fe640 1 -- 172.21.15.52:0/762329182 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730032810 con 0x7f073415f430 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.658+0000 7f073af46640 1 -- 172.21.15.52:0/762329182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f073415f430 msgr2=0x7f073415fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.658+0000 7f073af46640 1 --2- 172.21.15.52:0/762329182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f073415f430 0x7f073415fd70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f07300097a0 tx=0x7f073002a180 comp rx=0 tx=0).stop 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.658+0000 7f073af46640 1 -- 172.21.15.52:0/762329182 shutdown_connections 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.658+0000 7f073af46640 1 --2- 172.21.15.52:0/762329182 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 0x7f07340d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.658+0000 7f073af46640 1 --2- 172.21.15.52:0/762329182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f073415f430 0x7f073415fd70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.658+0000 7f073af46640 1 -- 172.21.15.52:0/762329182 >> 172.21.15.52:0/762329182 conn(0x7f073408eb70 msgr2=0x7f073409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.658+0000 7f073af46640 1 -- 172.21.15.52:0/762329182 shutdown_connections 2024-05-11T01:01:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.658+0000 7f073af46640 1 -- 172.21.15.52:0/762329182 wait complete. 2024-05-11T01:01:34.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.659+0000 7f073af46640 1 Processor -- start 2024-05-11T01:01:34.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.659+0000 7f073af46640 1 -- start start 2024-05-11T01:01:34.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f073af46640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 0x7f07340b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f073af46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073415f430 0x7f0734155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f073af46640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07341602b0 0x7f07340d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f073af46640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0734163030 con 0x7f07341602b0 2024-05-11T01:01:34.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f073af46640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0734162eb0 con 0x7f073415f430 2024-05-11T01:01:34.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f073af46640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f07340d27a0 con 0x7f07340d3d90 2024-05-11T01:01:34.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f07394e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07341602b0 0x7f07340d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f07394e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07341602b0 0x7f07340d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47242/0 (socket says 172.21.15.52:47242) 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f07394e3640 1 -- 172.21.15.52:0/3972214029 learned_addr learned my addr 172.21.15.52:0/3972214029 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f0738ce2640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 0x7f07340b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f072bfff640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073415f430 0x7f0734155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f07394e3640 1 -- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 msgr2=0x7f07340b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f07394e3640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 0x7f07340b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f07394e3640 1 -- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073415f430 msgr2=0x7f0734155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f07394e3640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073415f430 0x7f0734155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f07394e3640 1 -- 172.21.15.52:0/3972214029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07340ccc30 con 0x7f07341602b0 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f072bfff640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073415f430 0x7f0734155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.660+0000 7f07394e3640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07341602b0 0x7f07340d1b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f072c007f20 tx=0x7f072c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.661+0000 7f07297fa640 1 -- 172.21.15.52:0/3972214029 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c01a070 con 0x7f07341602b0 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.661+0000 7f07297fa640 1 -- 172.21.15.52:0/3972214029 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072c003b90 con 0x7f07341602b0 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.661+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07340ccf10 con 0x7f07341602b0 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.661+0000 7f07297fa640 1 -- 172.21.15.52:0/3972214029 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c004030 con 0x7f07341602b0 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.661+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07340cd4d0 con 0x7f07341602b0 2024-05-11T01:01:34.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.662+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0734069780 con 0x7f07341602b0 2024-05-11T01:01:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.662+0000 7f07297fa640 1 -- 172.21.15.52:0/3972214029 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f072c01d070 con 0x7f07341602b0 2024-05-11T01:01:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.663+0000 7f07297fa640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07200762d0 0x7f0720078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.663+0000 7f07297fa640 1 -- 172.21.15.52:0/3972214029 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f072c0977c0 con 0x7f07341602b0 2024-05-11T01:01:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.663+0000 7f0738ce2640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07200762d0 0x7f0720078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.664+0000 7f0738ce2640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07200762d0 0x7f0720078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0730034000 tx=0x7f073002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:34.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.667+0000 7f07297fa640 1 -- 172.21.15.52:0/3972214029 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f072c010070 con 0x7f07341602b0 2024-05-11T01:01:34.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.860+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07340b5d20 con 0x7f07341602b0 2024-05-11T01:01:34.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.861+0000 7f07297fa640 1 -- 172.21.15.52:0/3972214029 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f072c060b60 con 0x7f07341602b0 2024-05-11T01:01:34.862 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:34.862 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":20,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:02.673511+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:00:59.086890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:00.008780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:34.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07200762d0 msgr2=0x7f0720078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07200762d0 0x7f0720078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0730034000 tx=0x7f073002e000 comp rx=0 tx=0).stop 2024-05-11T01:01:34.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07341602b0 msgr2=0x7f07340d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07341602b0 0x7f07340d1b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f072c007f20 tx=0x7f072c00d560 comp rx=0 tx=0).stop 2024-05-11T01:01:34.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 shutdown_connections 2024-05-11T01:01:34.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07200762d0 0x7f0720078790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07341602b0 0x7f07340d1b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073415f430 0x7f0734155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 --2- 172.21.15.52:0/3972214029 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07340d3d90 0x7f07340b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 >> 172.21.15.52:0/3972214029 conn(0x7f073408eb70 msgr2=0x7f073406d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:34.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 shutdown_connections 2024-05-11T01:01:34.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.864+0000 7f073af46640 1 -- 172.21.15.52:0/3972214029 wait complete. 2024-05-11T01:01:34.875 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T01:01:34.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.979+0000 7f8a06e7c640 1 Processor -- start 2024-05-11T01:01:34.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.980+0000 7f8a06e7c640 1 -- start start 2024-05-11T01:01:34.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.980+0000 7f8a06e7c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a000b8a10 0x7f8a000d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.980+0000 7f8a06e7c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a000ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.980+0000 7f8a06e7c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a000cd6e0 con 0x7f8a000b55b0 2024-05-11T01:01:34.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.980+0000 7f8a06e7c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8a000ce2f0 con 0x7f8a000b8a10 2024-05-11T01:01:34.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.980+0000 7f8a06e7c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a000cd9c0 con 0x7f8a0009c130 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a05419640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a000ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a04c18640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a000b8a10 0x7f8a000d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a04c18640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a000b8a10 0x7f8a000d1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:54830/0 (socket says 172.21.15.52:54830) 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a04c18640 1 -- 172.21.15.52:0/186072355 learned_addr learned my addr 172.21.15.52:0/186072355 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a05419640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a000ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:49234/0 (socket says 172.21.15.52:49234) 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f89fffff640 1 --1- 172.21.15.52:0/186072355 >> v1:172.21.15.119:6789/0 conn(0x7f8a000b55b0 0x7f8a0009ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:42342/0 (socket says 172.21.15.52:42342) 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a04c18640 1 -- 172.21.15.52:0/186072355 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 msgr2=0x7f8a000ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a04c18640 1 --2- 172.21.15.52:0/186072355 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a000ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a04c18640 1 -- 172.21.15.52:0/186072355 >> v1:172.21.15.119:6789/0 conn(0x7f8a000b55b0 legacy=0x7f8a0009ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a04c18640 1 -- 172.21.15.52:0/186072355 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000cdca0 con 0x7f8a000b8a10 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f89ff7fe640 1 -- 172.21.15.52:0/186072355 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827446231 0 0) 0x7f8a000cd6e0 con 0x7f8a000b55b0 2024-05-11T01:01:34.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.981+0000 7f8a05419640 1 --2- 172.21.15.52:0/186072355 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a000ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.982+0000 7f8a04c18640 1 --2- 172.21.15.52:0/186072355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 0x7f8a000d1cf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009e50 tx=0x7f89f802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40de206beacdaab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.982+0000 7f89ff7fe640 1 -- 172.21.15.52:0/186072355 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8036070 con 0x7f8a000b8a10 2024-05-11T01:01:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.982+0000 7f89ff7fe640 1 -- 172.21.15.52:0/186072355 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f8031600 con 0x7f8a000b8a10 2024-05-11T01:01:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.982+0000 7f89ff7fe640 1 -- 172.21.15.52:0/186072355 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f80319d0 con 0x7f8a000b8a10 2024-05-11T01:01:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.982+0000 7f8a06e7c640 1 -- 172.21.15.52:0/186072355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 msgr2=0x7f8a000d1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.982+0000 7f8a06e7c640 1 --2- 172.21.15.52:0/186072355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 0x7f8a000d1cf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009e50 tx=0x7f89f802a680 comp rx=0 tx=0).stop 2024-05-11T01:01:34.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.983+0000 7f8a06e7c640 1 -- 172.21.15.52:0/186072355 shutdown_connections 2024-05-11T01:01:34.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.983+0000 7f8a06e7c640 1 --2- 172.21.15.52:0/186072355 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a000ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.983+0000 7f8a06e7c640 1 --2- 172.21.15.52:0/186072355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 0x7f8a000d1cf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.983+0000 7f8a06e7c640 1 -- 172.21.15.52:0/186072355 >> 172.21.15.52:0/186072355 conn(0x7f8a000b4a90 msgr2=0x7f8a000b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:34.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.983+0000 7f8a06e7c640 1 -- 172.21.15.52:0/186072355 shutdown_connections 2024-05-11T01:01:34.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.983+0000 7f8a06e7c640 1 -- 172.21.15.52:0/186072355 wait complete. 2024-05-11T01:01:34.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.984+0000 7f8a06e7c640 1 Processor -- start 2024-05-11T01:01:34.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.984+0000 7f8a06e7c640 1 -- start start 2024-05-11T01:01:34.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f8a06e7c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a001593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f8a06e7c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a000b55b0 0x7f8a00161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f8a06e7c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 0x7f8a000714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f8a06e7c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000b5db0 con 0x7f8a000b55b0 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f8a06e7c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000b8df0 con 0x7f8a000b8a10 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f8a06e7c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0006ee60 con 0x7f8a0009c130 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f89fffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a000b55b0 0x7f8a00161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f89fffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a000b55b0 0x7f8a00161340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47258/0 (socket says 172.21.15.52:47258) 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f89fffff640 1 -- 172.21.15.52:0/4141410227 learned_addr learned my addr 172.21.15.52:0/4141410227 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f8a05419640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 0x7f8a000714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f8a04c18640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a001593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f89fffff640 1 -- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 msgr2=0x7f8a001593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f89fffff640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a001593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f89fffff640 1 -- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 msgr2=0x7f8a000714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:34.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f89fffff640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 0x7f8a000714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:34.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f89fffff640 1 -- 172.21.15.52:0/4141410227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a00162d30 con 0x7f8a000b55b0 2024-05-11T01:01:34.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f8a05419640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 0x7f8a000714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:34.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.985+0000 7f89fffff640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a000b55b0 0x7f8a00161340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c3e0 tx=0x7f89f400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:34.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.986+0000 7f89fd7fa640 1 -- 172.21.15.52:0/4141410227 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f40074e0 con 0x7f8a000b55b0 2024-05-11T01:01:34.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.986+0000 7f89fd7fa640 1 -- 172.21.15.52:0/4141410227 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f40047e0 con 0x7f8a000b55b0 2024-05-11T01:01:34.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.986+0000 7f89fd7fa640 1 -- 172.21.15.52:0/4141410227 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4004bd0 con 0x7f8a000b55b0 2024-05-11T01:01:34.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.986+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a00161c90 con 0x7f8a000b55b0 2024-05-11T01:01:34.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.986+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a00162250 con 0x7f8a000b55b0 2024-05-11T01:01:34.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.987+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a00069780 con 0x7f8a000b55b0 2024-05-11T01:01:34.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.987+0000 7f89fd7fa640 1 -- 172.21.15.52:0/4141410227 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f89f4011070 con 0x7f8a000b55b0 2024-05-11T01:01:34.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.988+0000 7f89fd7fa640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89ec0762d0 0x7f89ec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:34.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.988+0000 7f89fd7fa640 1 -- 172.21.15.52:0/4141410227 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89f4097040 con 0x7f8a000b55b0 2024-05-11T01:01:34.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.988+0000 7f8a04c18640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89ec0762d0 0x7f89ec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:34.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.989+0000 7f8a04c18640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89ec0762d0 0x7f89ec078790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89f8033000 tx=0x7f89f8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:34.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:34.993+0000 7f89fd7fa640 1 -- 172.21.15.52:0/4141410227 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f89f40609a0 con 0x7f8a000b55b0 2024-05-11T01:01:35.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.222+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8a00159910 con 0x7f8a000b55b0 2024-05-11T01:01:35.223 INFO:tasks.ceph.mon.a.smithi052.stderr:2024-05-11T01:01:35.223+0000 7f9c6a575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T01:01:35.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.224+0000 7f89fd7fa640 1 -- 172.21.15.52:0/4141410227 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f89f40602a0 con 0x7f8a000b55b0 2024-05-11T01:01:35.224 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T01:01:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89ec0762d0 msgr2=0x7f89ec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89ec0762d0 0x7f89ec078790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89f8033000 tx=0x7f89f8009000 comp rx=0 tx=0).stop 2024-05-11T01:01:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a000b55b0 msgr2=0x7f8a00161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a000b55b0 0x7f8a00161340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c3e0 tx=0x7f89f400c7e0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 shutdown_connections 2024-05-11T01:01:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f89ec0762d0 0x7f89ec078790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a000b8a10 0x7f8a000714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a000b55b0 0x7f8a00161340 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 --2- 172.21.15.52:0/4141410227 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a0009c130 0x7f8a001593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 >> 172.21.15.52:0/4141410227 conn(0x7f8a000b4a90 msgr2=0x7f8a000b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 shutdown_connections 2024-05-11T01:01:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.227+0000 7f8a06e7c640 1 -- 172.21.15.52:0/4141410227 wait complete. 2024-05-11T01:01:35.229 INFO:tasks.ceph.mds.b.smithi119.stderr: -2011> 2024-05-11T01:01:06.800+0000 7fded0c78640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T01:01:35.240 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T01:01:35.260 INFO:tasks.ceph.mds.b.smithi119.stdout:starting mds.b at 2024-05-11T01:01:35.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.345+0000 7f07e35af640 1 Processor -- start 2024-05-11T01:01:35.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.346+0000 7f07e35af640 1 -- start start 2024-05-11T01:01:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.346+0000 7f07e35af640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f07dc0b3cf0 0x7f07dc0d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.346+0000 7f07e35af640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 0x7f07dc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.346+0000 7f07e35af640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07dc160890 con 0x7f07dc15cdd0 2024-05-11T01:01:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.346+0000 7f07e35af640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f07dc0d3850 con 0x7f07dc0b3cf0 2024-05-11T01:01:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.346+0000 7f07e35af640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc0b2c50 con 0x7f07dc0d33e0 2024-05-11T01:01:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07e134b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f07dc15cdd0 0x7f07dc160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51740/0 (socket says 172.21.15.52:51740) 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07e134b640 1 -- 172.21.15.52:0/714431266 learned_addr learned my addr 172.21.15.52:0/714431266 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07e0b4a640 1 --2- 172.21.15.52:0/714431266 >> v2:172.21.15.119:3300/0 conn(0x7f07dc0b3cf0 0x7f07dc0d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07e1b4c640 1 --2- 172.21.15.52:0/714431266 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 0x7f07dc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07dbfff640 1 -- 172.21.15.52:0/714431266 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 162421347 0 0) 0x7f07dc160890 con 0x7f07dc15cdd0 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07dbfff640 1 -- 172.21.15.52:0/714431266 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07c4003520 con 0x7f07dc15cdd0 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07e1b4c640 1 -- 172.21.15.52:0/714431266 >> v2:172.21.15.119:3300/0 conn(0x7f07dc0b3cf0 msgr2=0x7f07dc0d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07e1b4c640 1 --2- 172.21.15.52:0/714431266 >> v2:172.21.15.119:3300/0 conn(0x7f07dc0b3cf0 0x7f07dc0d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07dbfff640 1 -- 172.21.15.52:0/714431266 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1227535815 0 0) 0x7f07c4003520 con 0x7f07dc15cdd0 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07e1b4c640 1 -- 172.21.15.52:0/714431266 >> v1:172.21.15.52:6789/0 conn(0x7f07dc15cdd0 legacy=0x7f07dc160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07e1b4c640 1 -- 172.21.15.52:0/714431266 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc0b2f30 con 0x7f07dc0d33e0 2024-05-11T01:01:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.347+0000 7f07e0b4a640 1 --2- 172.21.15.52:0/714431266 >> v2:172.21.15.119:3300/0 conn(0x7f07dc0b3cf0 0x7f07dc0d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.348+0000 7f07e1b4c640 1 --2- 172.21.15.52:0/714431266 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 0x7f07dc06d590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009d90 tx=0x7f07cc02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a496e418ca5afa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.348+0000 7f07dbfff640 1 -- 172.21.15.52:0/714431266 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d40016f0 con 0x7f07dc0d33e0 2024-05-11T01:01:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.348+0000 7f07dbfff640 1 -- 172.21.15.52:0/714431266 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc007540 con 0x7f07dc0d33e0 2024-05-11T01:01:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.348+0000 7f07dbfff640 1 -- 172.21.15.52:0/714431266 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc007910 con 0x7f07dc0d33e0 2024-05-11T01:01:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.349+0000 7f07e35af640 1 -- 172.21.15.52:0/714431266 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 msgr2=0x7f07dc06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.349+0000 7f07e35af640 1 --2- 172.21.15.52:0/714431266 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 0x7f07dc06d590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009d90 tx=0x7f07cc02a5f0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.349+0000 7f07e35af640 1 -- 172.21.15.52:0/714431266 shutdown_connections 2024-05-11T01:01:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.349+0000 7f07e35af640 1 --2- 172.21.15.52:0/714431266 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 0x7f07dc06d590 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.349+0000 7f07e35af640 1 --2- 172.21.15.52:0/714431266 >> v2:172.21.15.119:3300/0 conn(0x7f07dc0b3cf0 0x7f07dc0d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.349+0000 7f07e35af640 1 -- 172.21.15.52:0/714431266 >> 172.21.15.52:0/714431266 conn(0x7f07dc09ba20 msgr2=0x7f07dc09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.349+0000 7f07e35af640 1 -- 172.21.15.52:0/714431266 shutdown_connections 2024-05-11T01:01:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.349+0000 7f07e35af640 1 -- 172.21.15.52:0/714431266 wait complete. 2024-05-11T01:01:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.350+0000 7f07e35af640 1 Processor -- start 2024-05-11T01:01:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.350+0000 7f07e35af640 1 -- start start 2024-05-11T01:01:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e35af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0b3cf0 0x7f07dc151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e35af640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 0x7f07dc0c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e35af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07dc15cdd0 0x7f07dc15e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e35af640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0b37d0 con 0x7f07dc15cdd0 2024-05-11T01:01:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e35af640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0b34f0 con 0x7f07dc0b3cf0 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e35af640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc15d1b0 con 0x7f07dc0d33e0 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e1b4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07dc15cdd0 0x7f07dc15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e1b4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07dc15cdd0 0x7f07dc15e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47262/0 (socket says 172.21.15.52:47262) 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e1b4c640 1 -- 172.21.15.52:0/3003263516 learned_addr learned my addr 172.21.15.52:0/3003263516 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e0b4a640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 0x7f07dc0c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e134b640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0b3cf0 0x7f07dc151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e1b4c640 1 -- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 msgr2=0x7f07dc0c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e1b4c640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 0x7f07dc0c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e1b4c640 1 -- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0b3cf0 msgr2=0x7f07dc151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e1b4c640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0b3cf0 0x7f07dc151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e1b4c640 1 -- 172.21.15.52:0/3003263516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc0c6fb0 con 0x7f07dc15cdd0 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07e1b4c640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07dc15cdd0 0x7f07dc15e470 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009eb0 tx=0x7f07cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.351+0000 7f07d9ffb640 1 -- 172.21.15.52:0/3003263516 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc040070 con 0x7f07dc15cdd0 2024-05-11T01:01:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.352+0000 7f07d9ffb640 1 -- 172.21.15.52:0/3003263516 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc031bc0 con 0x7f07dc15cdd0 2024-05-11T01:01:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.352+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc0ca230 con 0x7f07dc15cdd0 2024-05-11T01:01:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.352+0000 7f07d9ffb640 1 -- 172.21.15.52:0/3003263516 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc007e90 con 0x7f07dc15cdd0 2024-05-11T01:01:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.352+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc0c6bd0 con 0x7f07dc15cdd0 2024-05-11T01:01:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.353+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07dc0694f0 con 0x7f07dc15cdd0 2024-05-11T01:01:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.353+0000 7f07d9ffb640 1 -- 172.21.15.52:0/3003263516 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f07cc039040 con 0x7f07dc15cdd0 2024-05-11T01:01:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.354+0000 7f07d9ffb640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07c80762d0 0x7f07c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.354+0000 7f07d9ffb640 1 -- 172.21.15.52:0/3003263516 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f07cc0b88b0 con 0x7f07dc15cdd0 2024-05-11T01:01:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.354+0000 7f07e134b640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07c80762d0 0x7f07c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.355+0000 7f07e134b640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07c80762d0 0x7f07c8078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f07d4005880 tx=0x7f07d4001db0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.358+0000 7f07d9ffb640 1 -- 172.21.15.52:0/3003263516 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f07cc081b50 con 0x7f07dc15cdd0 2024-05-11T01:01:35.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:35.589+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f07dc0c7320 con 0x7f07dc15cdd0 2024-05-11T01:01:36.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.228+0000 7f07d9ffb640 1 -- 172.21.15.52:0/3003263516 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f07cc085e50 con 0x7f07dc15cdd0 2024-05-11T01:01:36.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07c80762d0 msgr2=0x7f07c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:36.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07c80762d0 0x7f07c8078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f07d4005880 tx=0x7f07d4001db0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07dc15cdd0 msgr2=0x7f07dc15e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07dc15cdd0 0x7f07dc15e470 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009eb0 tx=0x7f07cc004a40 comp rx=0 tx=0).stop 2024-05-11T01:01:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 shutdown_connections 2024-05-11T01:01:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f07c80762d0 0x7f07c8078790 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07dc15cdd0 0x7f07dc15e470 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f07dc0d33e0 0x7f07dc0c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 --2- 172.21.15.52:0/3003263516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0b3cf0 0x7f07dc151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 >> 172.21.15.52:0/3003263516 conn(0x7f07dc09ba20 msgr2=0x7f07dc0cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 shutdown_connections 2024-05-11T01:01:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.231+0000 7f07e35af640 1 -- 172.21.15.52:0/3003263516 wait complete. 2024-05-11T01:01:36.243 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T01:01:36.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.354+0000 7fb8221f0640 1 Processor -- start 2024-05-11T01:01:36.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.354+0000 7fb8221f0640 1 -- start start 2024-05-11T01:01:36.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.355+0000 7fb8221f0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb81c0d2590 0x7fb81c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:36.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.355+0000 7fb8221f0640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb81c0cfd40 con 0x7fb81c15f310 2024-05-11T01:01:36.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.355+0000 7fb8221f0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb81c0d29a0 con 0x7fb81c0d57a0 2024-05-11T01:01:36.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.355+0000 7fb8221f0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb81c0b5300 con 0x7fb81c0d2590 2024-05-11T01:01:36.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.355+0000 7fb81b7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb81c15f310 0x7fb81c0d5090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51752/0 (socket says 172.21.15.52:51752) 2024-05-11T01:01:36.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.355+0000 7fb81b7fe640 1 -- 172.21.15.52:0/3786119901 learned_addr learned my addr 172.21.15.52:0/3786119901 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:36.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.355+0000 7fb81bfff640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb81c0d2590 0x7fb81c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:36.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.355+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099679246 0 0) 0x7fb80c001110 con 0x7fb81c15f310 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.355+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb808003520 con 0x7fb81c15f310 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1341473657 0 0) 0x7fb81c0d29a0 con 0x7fb81c0d57a0 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb80c001110 con 0x7fb81c0d57a0 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1912232330 0 0) 0x7fb808003520 con 0x7fb81c15f310 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb81c0d2590 msgr2=0x7fb81c0b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb81c0d2590 0x7fb81c0b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 >> v1:172.21.15.119:6789/0 conn(0x7fb81c0d57a0 legacy=0x7fb81c0d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c0b5530 con 0x7fb81c15f310 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fb80c0018c0 con 0x7fb81c15f310 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 >> v1:172.21.15.52:6789/0 conn(0x7fb81c15f310 legacy=0x7fb81c0d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb808003c60 0x7fb808004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:36.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb81c0d2590 0x7fb808004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:36.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb808004d90 0x7fb808005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:36.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb80800bb50 con 0x7fb808004d90 2024-05-11T01:01:36.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb80800bd00 con 0x7fb808003c60 2024-05-11T01:01:36.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb80800bf00 con 0x7fb81c0d2590 2024-05-11T01:01:36.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81bfff640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb808004d90 0x7fb808005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81b7fe640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb808003c60 0x7fb808004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb813fff640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb81c0d2590 0x7fb808004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81bfff640 1 -- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb81c0d2590 msgr2=0x7fb808004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81bfff640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb81c0d2590 0x7fb808004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81bfff640 1 -- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb808003c60 msgr2=0x7fb808004060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81bfff640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb808003c60 0x7fb808004060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.356+0000 7fb81bfff640 1 -- 172.21.15.52:0/3786119901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb808003520 con 0x7fb808004d90 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.357+0000 7fb81bfff640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb808004d90 0x7fb808005210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb814009fc0 tx=0x7fb808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d8726d9d1fd7b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.357+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814009c10 con 0x7fb808004d90 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.357+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb814036070 con 0x7fb808004d90 2024-05-11T01:01:36.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.357+0000 7fb81affd640 1 -- 172.21.15.52:0/3786119901 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814031880 con 0x7fb808004d90 2024-05-11T01:01:36.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.358+0000 7fb8221f0640 1 -- 172.21.15.52:0/3786119901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb808004d90 msgr2=0x7fb808005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:36.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.358+0000 7fb8221f0640 1 --2- 172.21.15.52:0/3786119901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb808004d90 0x7fb808005210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb814009fc0 tx=0x7fb808005830 comp rx=0 tx=0).stop 2024-05-11T01:01:36.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.358+0000 7fb81bfff640 1 -- 172.21.15.52:0/3786119901 reap_dead start 2024-05-11T01:01:36.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.358+0000 7fb8221f0640 1 -- 172.21.15.52:0/3786119901 shutdown_connections 2024-05-11T01:01:36.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.358+0000 7fb8221f0640 1 -- 172.21.15.52:0/3786119901 >> 172.21.15.52:0/3786119901 conn(0x7fb81c08eb80 msgr2=0x7fb81c09e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:36.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.358+0000 7fb8221f0640 1 -- 172.21.15.52:0/3786119901 shutdown_connections 2024-05-11T01:01:36.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.358+0000 7fb8221f0640 1 -- 172.21.15.52:0/3786119901 wait complete. 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.359+0000 7fb8221f0640 1 Processor -- start 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.359+0000 7fb8221f0640 1 -- start start 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb8221f0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb808004d90 0x7fb81c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb8221f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb81c1f9720 0x7fb81c1f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb8221f0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb81c1f5a60 0x7fb81c152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb8221f0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c0b5af0 con 0x7fb81c1f9720 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb8221f0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c0b6140 con 0x7fb81c1f5a60 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb8221f0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb81c0b5dd0 con 0x7fb808004d90 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb813fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb81c1f9720 0x7fb81c1f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb81b7fe640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb808004d90 0x7fb81c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb813fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb81c1f9720 0x7fb81c1f5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47272/0 (socket says 172.21.15.52:47272) 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb813fff640 1 -- 172.21.15.52:0/2379581717 learned_addr learned my addr 172.21.15.52:0/2379581717 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb81bfff640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb81c1f5a60 0x7fb81c152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb813fff640 1 -- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb808004d90 msgr2=0x7fb81c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:36.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb813fff640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb808004d90 0x7fb81c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb813fff640 1 -- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb81c1f5a60 msgr2=0x7fb81c152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb813fff640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb81c1f5a60 0x7fb81c152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb813fff640 1 -- 172.21.15.52:0/2379581717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c1f6480 con 0x7fb81c1f9720 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb813fff640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb81c1f9720 0x7fb81c1f5520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb80400f890 tx=0x7fb80400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.360+0000 7fb818ff9640 1 -- 172.21.15.52:0/2379581717 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040052e0 con 0x7fb81c1f9720 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.361+0000 7fb818ff9640 1 -- 172.21.15.52:0/2379581717 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb804018070 con 0x7fb81c1f9720 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.361+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c1f5ee0 con 0x7fb81c1f9720 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.361+0000 7fb818ff9640 1 -- 172.21.15.52:0/2379581717 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80400d970 con 0x7fb81c1f9720 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.361+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c1f8c40 con 0x7fb81c1f9720 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.362+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81c069780 con 0x7fb81c1f9720 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.362+0000 7fb818ff9640 1 -- 172.21.15.52:0/2379581717 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb80400db10 con 0x7fb81c1f9720 2024-05-11T01:01:36.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.363+0000 7fb818ff9640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb7fc0762d0 0x7fb7fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:36.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.363+0000 7fb818ff9640 1 -- 172.21.15.52:0/2379581717 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7fb804097460 con 0x7fb81c1f9720 2024-05-11T01:01:36.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.363+0000 7fb81b7fe640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb7fc0762d0 0x7fb7fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:36.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.363+0000 7fb81b7fe640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb7fc0762d0 0x7fb7fc078790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0018a0 tx=0x7fb80c000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:36.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.367+0000 7fb818ff9640 1 -- 172.21.15.52:0/2379581717 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb804060df0 con 0x7fb81c1f9720 2024-05-11T01:01:36.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:36.557+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb81c069e10 con 0x7fb81c1f9720 2024-05-11T01:01:37.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.230+0000 7fb818ff9640 1 -- 172.21.15.52:0/2379581717 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7fb8040606f0 con 0x7fb81c1f9720 2024-05-11T01:01:37.230 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-05-11T01:01:37.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb7fc0762d0 msgr2=0x7fb7fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:37.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb7fc0762d0 0x7fb7fc078790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0018a0 tx=0x7fb80c000ed0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb81c1f9720 msgr2=0x7fb81c1f5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:37.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb81c1f9720 0x7fb81c1f5520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb80400f890 tx=0x7fb80400fc90 comp rx=0 tx=0).stop 2024-05-11T01:01:37.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 shutdown_connections 2024-05-11T01:01:37.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb7fc0762d0 0x7fb7fc078790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb81c1f5a60 0x7fb81c152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb81c1f9720 0x7fb81c1f5520 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 --2- 172.21.15.52:0/2379581717 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb808004d90 0x7fb81c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.234+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 >> 172.21.15.52:0/2379581717 conn(0x7fb81c08eb80 msgr2=0x7fb81c06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:37.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.235+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 shutdown_connections 2024-05-11T01:01:37.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.235+0000 7fb8221f0640 1 -- 172.21.15.52:0/2379581717 wait complete. 2024-05-11T01:01:37.249 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T01:01:37.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.359+0000 7feea2ddd640 1 Processor -- start 2024-05-11T01:01:37.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.359+0000 7feea2ddd640 1 -- start start 2024-05-11T01:01:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.360+0000 7feea2ddd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee9c15f690 0x7fee9c0d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.360+0000 7feea2ddd640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9c0d09b0 con 0x7fee9c0d56f0 2024-05-11T01:01:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.360+0000 7feea2ddd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9c0d0c90 con 0x7fee9c0d3380 2024-05-11T01:01:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.360+0000 7feea2ddd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fee9c0b50a0 con 0x7fee9c15f690 2024-05-11T01:01:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.360+0000 7feea137a640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fee9c0d56f0 0x7fee9c0b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51766/0 (socket says 172.21.15.52:51766) 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.360+0000 7feea137a640 1 -- 172.21.15.52:0/2618149926 learned_addr learned my addr 172.21.15.52:0/2618149926 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.360+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2552662822 0 0) 0x7fee9c0d09b0 con 0x7fee9c0d56f0 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee84003520 con 0x7fee9c0d56f0 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7feea0b79640 1 --2- 172.21.15.52:0/2618149926 >> v2:172.21.15.119:3300/0 conn(0x7fee9c15f690 0x7fee9c0d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 250348465 0 0) 0x7fee84003520 con 0x7fee9c0d56f0 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 >> v2:172.21.15.119:3300/0 conn(0x7fee9c15f690 msgr2=0x7fee9c0d3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 --2- 172.21.15.52:0/2618149926 >> v2:172.21.15.119:3300/0 conn(0x7fee9c15f690 0x7fee9c0d3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 >> v1:172.21.15.119:6789/0 conn(0x7fee9c0d3380 legacy=0x7fee9c0d4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c0d0f70 con 0x7fee9c0d56f0 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fee94001a70 con 0x7fee9c0d56f0 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 >> v1:172.21.15.52:6789/0 conn(0x7fee9c0d56f0 legacy=0x7fee9c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84004800 0x7fee84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee840051c0 0x7fee8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:37.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0b50a0 con 0x7fee84003a00 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee8400c060 con 0x7fee84004800 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fee8400c260 con 0x7fee840051c0 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7feea0b79640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9bfff640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84004800 0x7fee84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7feea137a640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee840051c0 0x7fee8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7feea0b79640 1 -- 172.21.15.52:0/2618149926 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee840051c0 msgr2=0x7fee8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7feea0b79640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee840051c0 0x7fee8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7feea0b79640 1 -- 172.21.15.52:0/2618149926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84004800 msgr2=0x7fee84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7feea0b79640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84004800 0x7fee84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7feea0b79640 1 -- 172.21.15.52:0/2618149926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee84003520 con 0x7fee84003a00 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7feea137a640 1 -- 172.21.15.52:0/2618149926 reap_dead start 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.361+0000 7fee9bfff640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84004800 0x7fee84004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.362+0000 7feea0b79640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee84003de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009ac0 tx=0x7fee84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f58efeb52a4a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.362+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c036070 con 0x7fee84003a00 2024-05-11T01:01:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.362+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c031400 con 0x7fee84003a00 2024-05-11T01:01:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.362+0000 7fee9b7fe640 1 -- 172.21.15.52:0/2618149926 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c009690 con 0x7fee84003a00 2024-05-11T01:01:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.363+0000 7feea2ddd640 1 -- 172.21.15.52:0/2618149926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 msgr2=0x7fee84003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.363+0000 7feea2ddd640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee84003de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009ac0 tx=0x7fee84005830 comp rx=0 tx=0).stop 2024-05-11T01:01:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.363+0000 7feea2ddd640 1 -- 172.21.15.52:0/2618149926 shutdown_connections 2024-05-11T01:01:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.363+0000 7feea2ddd640 1 --2- 172.21.15.52:0/2618149926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee84003de0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.363+0000 7feea2ddd640 1 -- 172.21.15.52:0/2618149926 >> 172.21.15.52:0/2618149926 conn(0x7fee9c08ebf0 msgr2=0x7fee9c09e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.363+0000 7feea2ddd640 1 -- 172.21.15.52:0/2618149926 shutdown_connections 2024-05-11T01:01:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.363+0000 7feea2ddd640 1 -- 172.21.15.52:0/2618149926 wait complete. 2024-05-11T01:01:37.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.364+0000 7feea2ddd640 1 Processor -- start 2024-05-11T01:01:37.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.364+0000 7feea2ddd640 1 -- start start 2024-05-11T01:01:37.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea2ddd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee9c1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:37.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea2ddd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee9c15d270 0x7fee9c0ba540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:37.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea2ddd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee9c0baa80 0x7fee9c0cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:37.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea2ddd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0d4fe0 con 0x7fee84003a00 2024-05-11T01:01:37.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea2ddd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c15fa70 con 0x7fee9c0baa80 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea2ddd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c0d5160 con 0x7fee9c15d270 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea0b79640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee9c1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea0b79640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee9c1617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47298/0 (socket says 172.21.15.52:47298) 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea0b79640 1 -- 172.21.15.52:0/2267855628 learned_addr learned my addr 172.21.15.52:0/2267855628 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea137a640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee9c0baa80 0x7fee9c0cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7fee9bfff640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee9c15d270 0x7fee9c0ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea0b79640 1 -- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee9c15d270 msgr2=0x7fee9c0ba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea0b79640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee9c15d270 0x7fee9c0ba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea0b79640 1 -- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee9c0baa80 msgr2=0x7fee9c0cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea0b79640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee9c0baa80 0x7fee9c0cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea0b79640 1 -- 172.21.15.52:0/2267855628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c160730 con 0x7fee84003a00 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7fee9bfff640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee9c15d270 0x7fee9c0ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.365+0000 7feea0b79640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee9c1617d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee8c034040 tx=0x7fee8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:37.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.366+0000 7fee997fa640 1 -- 172.21.15.52:0/2267855628 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c03f020 con 0x7fee84003a00 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.366+0000 7fee997fa640 1 -- 172.21.15.52:0/2267855628 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c037040 con 0x7fee84003a00 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.366+0000 7fee997fa640 1 -- 172.21.15.52:0/2267855628 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c036070 con 0x7fee84003a00 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.366+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9c160a10 con 0x7fee84003a00 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.366+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9c160fd0 con 0x7fee84003a00 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.367+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9c069780 con 0x7fee84003a00 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.367+0000 7fee997fa640 1 -- 172.21.15.52:0/2267855628 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fee8c042020 con 0x7fee84003a00 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.368+0000 7fee997fa640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fee880762d0 0x7fee88078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.368+0000 7fee997fa640 1 -- 172.21.15.52:0/2267855628 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5357+0+0 (secure 0 0 0) 0x7fee8c0b73a0 con 0x7fee84003a00 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.368+0000 7fee9bfff640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fee880762d0 0x7fee88078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:37.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.369+0000 7fee9bfff640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fee880762d0 0x7fee88078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0cbb60 tx=0x7fee90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:37.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.372+0000 7fee997fa640 1 -- 172.21.15.52:0/2267855628 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fee8c049030 con 0x7fee84003a00 2024-05-11T01:01:37.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:37.565+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fee9c0b68e0 con 0x7fee84003a00 2024-05-11T01:01:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.235+0000 7fee997fa640 1 -- 172.21.15.52:0/2267855628 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7fee8c080e50 con 0x7fee84003a00 2024-05-11T01:01:38.235 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-05-11T01:01:38.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.241+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fee880762d0 msgr2=0x7fee88078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:38.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.241+0000 7feea2ddd640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fee880762d0 0x7fee88078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0cbb60 tx=0x7fee90009040 comp rx=0 tx=0).stop 2024-05-11T01:01:38.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.241+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 msgr2=0x7fee9c1617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:38.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.241+0000 7feea2ddd640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee9c1617d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee8c034040 tx=0x7fee8c004730 comp rx=0 tx=0).stop 2024-05-11T01:01:38.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.242+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 shutdown_connections 2024-05-11T01:01:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.242+0000 7feea2ddd640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fee880762d0 0x7fee88078790 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.242+0000 7feea2ddd640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee9c0baa80 0x7fee9c0cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.242+0000 7feea2ddd640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fee9c15d270 0x7fee9c0ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.242+0000 7feea2ddd640 1 --2- 172.21.15.52:0/2267855628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee84003a00 0x7fee9c1617d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:38.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.242+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 >> 172.21.15.52:0/2267855628 conn(0x7fee9c08ebf0 msgr2=0x7fee9c06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.243+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 shutdown_connections 2024-05-11T01:01:38.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.243+0000 7feea2ddd640 1 -- 172.21.15.52:0/2267855628 wait complete. 2024-05-11T01:01:38.259 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T01:01:38.260 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-11T01:01:38.260 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T01:01:38.260 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:01:38.260 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T01:01:38.260 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-11T01:01:38.260 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T01:01:38.260 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:01:38.261 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T01:01:38.261 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-11T01:01:38.261 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T01:01:38.261 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:01:38.261 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T01:01:38.261 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-11T01:01:38.262 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T01:01:38.262 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:01:38.272 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T01:01:38.272 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T01:01:38.273 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T01:01:38.274 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T01:01:38.275 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T01:01:38.276 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T01:01:38.277 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T01:01:38.277 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T01:01:38.315 INFO:tasks.ceph.mds.a:Started 2024-05-11T01:01:38.316 INFO:tasks.ceph.mds.c:Started 2024-05-11T01:01:38.317 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:38.317 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:38.320 INFO:tasks.ceph.mds.b:Started 2024-05-11T01:01:38.320 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:38.321 INFO:tasks.ceph.mds.d:Started 2024-05-11T01:01:38.321 DEBUG:teuthology.parallel:result is None 2024-05-11T01:01:38.321 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-11T01:01:38.399 INFO:tasks.ceph.mds.a.smithi052.stdout:starting mds.a at 2024-05-11T01:01:38.399 INFO:tasks.ceph.mds.c.smithi052.stdout:starting mds.c at 2024-05-11T01:01:38.401 INFO:tasks.ceph.mds.b.smithi119.stdout:starting mds.b at 2024-05-11T01:01:38.401 INFO:tasks.ceph.mds.d.smithi119.stdout:starting mds.d at 2024-05-11T01:01:38.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.436+0000 7fc61e578640 1 Processor -- start 2024-05-11T01:01:38.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.436+0000 7fc61e578640 1 -- start start 2024-05-11T01:01:38.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.436+0000 7fc61e578640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 0x7fc618160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:38.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.436+0000 7fc61e578640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6180d0d80 0x7fc6180d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:38.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc61e578640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6180d02e0 con 0x7fc6180b1260 2024-05-11T01:01:38.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc61e578640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc6180d08f0 con 0x7fc6180d0d80 2024-05-11T01:01:38.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc61e578640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc6180c7af0 con 0x7fc6180c7690 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc617fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc6180b1260 0x7fc61806fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51788/0 (socket says 172.21.15.52:51788) 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc617fff640 1 -- 172.21.15.52:0/3476591912 learned_addr learned my addr 172.21.15.52:0/3476591912 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc61cb15640 1 --2- 172.21.15.52:0/3476591912 >> v2:172.21.15.119:3300/0 conn(0x7fc6180d0d80 0x7fc6180d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc6177fe640 1 --2- 172.21.15.52:0/3476591912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 0x7fc618160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc616ffd640 1 -- 172.21.15.52:0/3476591912 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2788809832 0 0) 0x7fc6180d02e0 con 0x7fc6180b1260 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc616ffd640 1 -- 172.21.15.52:0/3476591912 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc600003520 con 0x7fc6180b1260 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc61cb15640 1 -- 172.21.15.52:0/3476591912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 msgr2=0x7fc618160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc61cb15640 1 --2- 172.21.15.52:0/3476591912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 0x7fc618160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc61cb15640 1 -- 172.21.15.52:0/3476591912 >> v1:172.21.15.52:6789/0 conn(0x7fc6180b1260 legacy=0x7fc61806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.437+0000 7fc61cb15640 1 -- 172.21.15.52:0/3476591912 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6180c7d50 con 0x7fc6180d0d80 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.438+0000 7fc61cb15640 1 --2- 172.21.15.52:0/3476591912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6180d0d80 0x7fc6180d1210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc610009d90 tx=0x7fc61002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93de17d8e46952f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.438+0000 7fc616ffd640 1 -- 172.21.15.52:0/3476591912 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610007740 con 0x7fc6180d0d80 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.438+0000 7fc616ffd640 1 -- 172.21.15.52:0/3476591912 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6100078e0 con 0x7fc6180d0d80 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.438+0000 7fc616ffd640 1 -- 172.21.15.52:0/3476591912 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610007cb0 con 0x7fc6180d0d80 2024-05-11T01:01:38.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.439+0000 7fc61e578640 1 -- 172.21.15.52:0/3476591912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6180d0d80 msgr2=0x7fc6180d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:38.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.439+0000 7fc61e578640 1 --2- 172.21.15.52:0/3476591912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6180d0d80 0x7fc6180d1210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc610009d90 tx=0x7fc61002a660 comp rx=0 tx=0).stop 2024-05-11T01:01:38.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.439+0000 7fc61e578640 1 -- 172.21.15.52:0/3476591912 shutdown_connections 2024-05-11T01:01:38.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.439+0000 7fc61e578640 1 --2- 172.21.15.52:0/3476591912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6180d0d80 0x7fc6180d1210 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:38.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.439+0000 7fc61e578640 1 --2- 172.21.15.52:0/3476591912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 0x7fc618160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:38.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.439+0000 7fc61e578640 1 -- 172.21.15.52:0/3476591912 >> 172.21.15.52:0/3476591912 conn(0x7fc6180b3a10 msgr2=0x7fc6180b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:38.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.439+0000 7fc61e578640 1 -- 172.21.15.52:0/3476591912 shutdown_connections 2024-05-11T01:01:38.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.439+0000 7fc61e578640 1 -- 172.21.15.52:0/3476591912 wait complete. 2024-05-11T01:01:38.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.440+0000 7fc61e578640 1 Processor -- start 2024-05-11T01:01:38.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.440+0000 7fc61e578640 1 -- start start 2024-05-11T01:01:38.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc61e578640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6180b1260 0x7fc6180b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:38.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc61e578640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 0x7fc61814a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:38.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc61e578640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6180d0d80 0x7fc61809ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:38.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc61e578640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6180d0580 con 0x7fc6180b1260 2024-05-11T01:01:38.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc61e578640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc618160c70 con 0x7fc6180d0d80 2024-05-11T01:01:38.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc61e578640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc618070470 con 0x7fc6180c7690 2024-05-11T01:01:38.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc617fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6180b1260 0x7fc6180b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:38.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc617fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6180b1260 0x7fc6180b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47318/0 (socket says 172.21.15.52:47318) 2024-05-11T01:01:38.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc617fff640 1 -- 172.21.15.52:0/1112760208 learned_addr learned my addr 172.21.15.52:0/1112760208 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:38.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc6177fe640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 0x7fc61814a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:38.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc61cb15640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6180d0d80 0x7fc61809ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:38.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc617fff640 1 -- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 msgr2=0x7fc61814a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:38.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc617fff640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 0x7fc61814a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:38.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc617fff640 1 -- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6180d0d80 msgr2=0x7fc61809ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:38.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc617fff640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6180d0d80 0x7fc61809ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:38.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.441+0000 7fc617fff640 1 -- 172.21.15.52:0/1112760208 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61809dec0 con 0x7fc6180b1260 2024-05-11T01:01:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.442+0000 7fc617fff640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6180b1260 0x7fc6180b6f80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc60800c440 tx=0x7fc60800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.442+0000 7fc614ff9640 1 -- 172.21.15.52:0/1112760208 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608017070 con 0x7fc6180b1260 2024-05-11T01:01:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.442+0000 7fc614ff9640 1 -- 172.21.15.52:0/1112760208 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc608005ce0 con 0x7fc6180b1260 2024-05-11T01:01:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.442+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61809e1a0 con 0x7fc6180b1260 2024-05-11T01:01:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.442+0000 7fc614ff9640 1 -- 172.21.15.52:0/1112760208 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608007270 con 0x7fc6180b1260 2024-05-11T01:01:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.442+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61809ea40 con 0x7fc6180b1260 2024-05-11T01:01:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.443+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc618069780 con 0x7fc6180b1260 2024-05-11T01:01:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.444+0000 7fc614ff9640 1 -- 172.21.15.52:0/1112760208 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc60801a070 con 0x7fc6180b1260 2024-05-11T01:01:38.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.444+0000 7fc614ff9640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc6040762d0 0x7fc604078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:38.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.444+0000 7fc614ff9640 1 -- 172.21.15.52:0/1112760208 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4943+0+0 (secure 0 0 0) 0x7fc6080964f0 con 0x7fc6180b1260 2024-05-11T01:01:38.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.444+0000 7fc6177fe640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc6040762d0 0x7fc604078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:38.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.445+0000 7fc6177fe640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc6040762d0 0x7fc604078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6181f6570 tx=0x7fc60c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:38.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.449+0000 7fc614ff9640 1 -- 172.21.15.52:0/1112760208 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc60805fb00 con 0x7fc6180b1260 2024-05-11T01:01:38.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:38.655+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc61806f630 con 0x7fc6180b1260 2024-05-11T01:01:39.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.249+0000 7fc614ff9640 1 -- 172.21.15.52:0/1112760208 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) ==== 97+0+0 (secure 0 0 0) 0x7fc608063e00 con 0x7fc6180b1260 2024-05-11T01:01:39.249 INFO:teuthology.orchestra.run.smithi052.stderr: removed all blocklist entries 2024-05-11T01:01:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc6040762d0 msgr2=0x7fc604078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc6040762d0 0x7fc604078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6181f6570 tx=0x7fc60c008040 comp rx=0 tx=0).stop 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6180b1260 msgr2=0x7fc6180b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6180b1260 0x7fc6180b6f80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc60800c440 tx=0x7fc60800c900 comp rx=0 tx=0).stop 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 shutdown_connections 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc6040762d0 0x7fc604078790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6180d0d80 0x7fc61809ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc6180c7690 0x7fc61814a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 --2- 172.21.15.52:0/1112760208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6180b1260 0x7fc6180b6f80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 >> 172.21.15.52:0/1112760208 conn(0x7fc6180b3a10 msgr2=0x7fc6180b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 shutdown_connections 2024-05-11T01:01:39.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.252+0000 7fc61e578640 1 -- 172.21.15.52:0/1112760208 wait complete. 2024-05-11T01:01:39.266 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-11T01:01:39.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.373+0000 7f3e9d3f0640 1 Processor -- start 2024-05-11T01:01:39.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.374+0000 7f3e9d3f0640 1 -- start start 2024-05-11T01:01:39.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.374+0000 7f3e9d3f0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3e9815f4b0 0x7f3e98070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.374+0000 7f3e9d3f0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3e980cc590 0x7f3e980cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.374+0000 7f3e9d3f0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e9806fb40 con 0x7f3e98071870 2024-05-11T01:01:39.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.374+0000 7f3e9d3f0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f3e98071120 con 0x7f3e9815f4b0 2024-05-11T01:01:39.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.374+0000 7f3e9d3f0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3e980ccf20 con 0x7f3e980cc590 2024-05-11T01:01:39.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.374+0000 7f3e96ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3e9815f4b0 0x7f3e98070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.375+0000 7f3e96ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3e9815f4b0 0x7f3e98070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47332/0 (socket says 172.21.15.52:47332) 2024-05-11T01:01:39.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.375+0000 7f3e96ffd640 1 -- 172.21.15.52:0/1603019984 learned_addr learned my addr 172.21.15.52:0/1603019984 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:39.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.375+0000 7f3e977fe640 1 --2- 172.21.15.52:0/1603019984 >> v2:172.21.15.119:3300/0 conn(0x7f3e980cc590 0x7f3e980cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.375+0000 7f3e96ffd640 1 -- 172.21.15.52:0/1603019984 >> v2:172.21.15.119:3300/0 conn(0x7f3e980cc590 msgr2=0x7f3e980cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:39.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.375+0000 7f3e96ffd640 1 --2- 172.21.15.52:0/1603019984 >> v2:172.21.15.119:3300/0 conn(0x7f3e980cc590 0x7f3e980cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.375+0000 7f3e96ffd640 1 -- 172.21.15.52:0/1603019984 >> v1:172.21.15.119:6789/0 conn(0x7f3e98071870 legacy=0x7f3e9806f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:39.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.375+0000 7f3e96ffd640 1 -- 172.21.15.52:0/1603019984 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9806fe20 con 0x7f3e9815f4b0 2024-05-11T01:01:39.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e96ffd640 1 --2- 172.21.15.52:0/1603019984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 0x7f3e98070be0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e980cc520 tx=0x7f3e9002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c87cb9fbc9ff51 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:39.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e95ffb640 1 -- 172.21.15.52:0/1603019984 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90036070 con 0x7f3e9815f4b0 2024-05-11T01:01:39.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e95ffb640 1 -- 172.21.15.52:0/1603019984 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e90031600 con 0x7f3e9815f4b0 2024-05-11T01:01:39.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e95ffb640 1 -- 172.21.15.52:0/1603019984 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900319d0 con 0x7f3e9815f4b0 2024-05-11T01:01:39.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1603019984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 msgr2=0x7f3e98070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e9d3f0640 1 --2- 172.21.15.52:0/1603019984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 0x7f3e98070be0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e980cc520 tx=0x7f3e9002a730 comp rx=0 tx=0).stop 2024-05-11T01:01:39.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1603019984 shutdown_connections 2024-05-11T01:01:39.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e9d3f0640 1 --2- 172.21.15.52:0/1603019984 >> v2:172.21.15.119:3300/0 conn(0x7f3e980cc590 0x7f3e980cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e9d3f0640 1 --2- 172.21.15.52:0/1603019984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 0x7f3e98070be0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1603019984 >> 172.21.15.52:0/1603019984 conn(0x7f3e980b3a30 msgr2=0x7f3e980b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:39.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1603019984 shutdown_connections 2024-05-11T01:01:39.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.376+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1603019984 wait complete. 2024-05-11T01:01:39.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.378+0000 7f3e9d3f0640 1 Processor -- start 2024-05-11T01:01:39.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.378+0000 7f3e9d3f0640 1 -- start start 2024-05-11T01:01:39.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.378+0000 7f3e9d3f0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e98071870 0x7f3e981611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.378+0000 7f3e9d3f0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e980cc590 0x7f3e980c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.378+0000 7f3e9d3f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 0x7f3e980c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.378+0000 7f3e9d3f0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e980706c0 con 0x7f3e9815f4b0 2024-05-11T01:01:39.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.378+0000 7f3e9d3f0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e980703e0 con 0x7f3e980cc590 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.378+0000 7f3e9d3f0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9815f890 con 0x7f3e98071870 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 0x7f3e980c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 0x7f3e980c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47338/0 (socket says 172.21.15.52:47338) 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e977fe640 1 -- 172.21.15.52:0/1969076080 learned_addr learned my addr 172.21.15.52:0/1969076080 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e967fc640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e980cc590 0x7f3e980c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e96ffd640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e98071870 0x7f3e981611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e977fe640 1 -- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e98071870 msgr2=0x7f3e981611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e977fe640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e98071870 0x7f3e981611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e977fe640 1 -- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e980cc590 msgr2=0x7f3e980c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e977fe640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e980cc590 0x7f3e980c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e977fe640 1 -- 172.21.15.52:0/1969076080 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e980c8d00 con 0x7f3e9815f4b0 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e977fe640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 0x7f3e980c9600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800d800 tx=0x7f3e8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e8f7fe640 1 -- 172.21.15.52:0/1969076080 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8800b6c0 con 0x7f3e9815f4b0 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e8f7fe640 1 -- 172.21.15.52:0/1969076080 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e88019070 con 0x7f3e9815f4b0 2024-05-11T01:01:39.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e98160140 con 0x7f3e9815f4b0 2024-05-11T01:01:39.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e8f7fe640 1 -- 172.21.15.52:0/1969076080 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8800bac0 con 0x7f3e9815f4b0 2024-05-11T01:01:39.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.379+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e98160700 con 0x7f3e9815f4b0 2024-05-11T01:01:39.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.380+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e98069670 con 0x7f3e9815f4b0 2024-05-11T01:01:39.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.381+0000 7f3e8f7fe640 1 -- 172.21.15.52:0/1969076080 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3e8800bc60 con 0x7f3e9815f4b0 2024-05-11T01:01:39.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.381+0000 7f3e8f7fe640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3e780762d0 0x7f3e78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.381+0000 7f3e8f7fe640 1 -- 172.21.15.52:0/1969076080 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f3e880977c0 con 0x7f3e9815f4b0 2024-05-11T01:01:39.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.382+0000 7f3e96ffd640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3e780762d0 0x7f3e78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.382+0000 7f3e96ffd640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3e780762d0 0x7f3e78078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3e90033000 tx=0x7f3e90009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:39.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.386+0000 7f3e8f7fe640 1 -- 172.21.15.52:0/1969076080 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3e880613d0 con 0x7f3e9815f4b0 2024-05-11T01:01:39.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.658+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3e980713d0 con 0x7f3e9815f4b0 2024-05-11T01:01:39.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.658+0000 7f3e8f7fe640 1 -- 172.21.15.52:0/1969076080 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f3e88060cd0 con 0x7f3e9815f4b0 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: "auth_dump": [ 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.a", 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mP3rNEhAAP5nXdayem1rp+TLkIpVbXg==", 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:01:39.659 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.b", 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5m7H/HHhAAFque3IN7APQjExckFVQiQg==", 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.c", 2024-05-11T01:01:39.660 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mmQINGhAAyEyoarOUTjK64GfK5PZ5bQ==", 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.d", 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mXGYNJhAA9loGi/oMhZT+LWktBsVhfw==", 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:01:39.661 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:01:39.662 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.662 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.662 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.662 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.662 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.0", 2024-05-11T01:01:39.662 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAfwz5mJZqrDxAAq3xCv8Pf2DPOEL2Tovrbeg==", 2024-05-11T01:01:39.662 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.662 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.1", 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAiwz5mkbxtDhAAHD/TxmE4UfGOWvArcQsdNA==", 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.663 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.2", 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAlwz5mgOegDBAAKU+gnyyeIwLuvXRfzOD/ZQ==", 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.3", 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAowz5mERnWChAAZVaafqVtHNgrii/OZSYXdg==", 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:01:39.664 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.4", 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAuwz5mbL3bERAAO7anuNWH7NtNl5//BukL8g==", 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.5", 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAxwz5myrFsEBAAvGSIqmyuovYeRCSoDNTLxg==", 2024-05-11T01:01:39.665 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.6", 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA0wz5mythuDxAAJh/fQNHjv0kVV420jaSiyQ==", 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.666 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.7", 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA3wz5mSAXGDRAAoWVOOP2dLYGaGsue3vxWwA==", 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.0", 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mUO0yKhAAfS2pcT6nSB+NrJccryMO8A==", 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:01:39.667 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.1", 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5m3AodMxAAXArXFLqTR6kBG8KTzop93w==", 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.2", 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5maFjoLxAAwD/wgFjjG9lbeRLUtV4BRQ==", 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.3", 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mEb+wOBAAI8i4rWMD8GWMUf6p0HICRA==", 2024-05-11T01:01:39.669 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.admin", 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAcwz5mmFmRMhAAIcbYdP/83D06BB/z3mJCTw==", 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:01:39.670 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-mds", 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA9wz5m4Wh1BRAAoa+EPVFRdqjOCjJyPV47Ig==", 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-mds" 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-mgr", 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA9wz5mEXF1BRAAEKWH2qa6xPlvuvU2B993GA==", 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-11T01:01:39.671 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-osd", 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA9wz5mBHh1BRAACnwfLLyJymL5lfztdOQJRg==", 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-osd" 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-rbd", 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA9wz5m9n51BRAALZCEkemJZ9qK6OweEDGHBQ==", 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.672 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA9wz5mV4Z1BRAAIbIPFzHGkusfBjpm7bUdrw==", 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.bootstrap-rgw", 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA9wz5mu411BRAAxGxNd6YN+aokiKWyN9r8rw==", 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mgr.x", 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mj0l4DxAAnDj4+dETpUu/ykNczo5dvA==", 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile mgr", 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mgr.y", 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5medHbCxAASin8Gv729ZztriQmuTTTYQ==", 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-05-11T01:01:39.674 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile mgr", 2024-05-11T01:01:39.675 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:01:39.675 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.675 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:39.675 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-05-11T01:01:39.675 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:01:39.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3e780762d0 msgr2=0x7f3e78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3e780762d0 0x7f3e78078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3e90033000 tx=0x7f3e90009000 comp rx=0 tx=0).stop 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 msgr2=0x7f3e980c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 0x7f3e980c9600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800d800 tx=0x7f3e8800dcc0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 shutdown_connections 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f3e780762d0 0x7f3e78078790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e9815f4b0 0x7f3e980c9600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e980cc590 0x7f3e980c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 --2- 172.21.15.52:0/1969076080 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e98071870 0x7f3e981611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 >> 172.21.15.52:0/1969076080 conn(0x7f3e980b3a30 msgr2=0x7f3e980b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 shutdown_connections 2024-05-11T01:01:39.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.660+0000 7f3e9d3f0640 1 -- 172.21.15.52:0/1969076080 wait complete. 2024-05-11T01:01:39.678 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-11T01:01:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.783+0000 7fd9424da640 1 Processor -- start 2024-05-11T01:01:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.783+0000 7fd9424da640 1 -- start start 2024-05-11T01:01:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd9424da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 0x7fd93c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd9424da640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd93c0d4ee0 0x7fd93c0d5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd9424da640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93c0b5bb0 con 0x7fd93c0b4e80 2024-05-11T01:01:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd9424da640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd93c0d3170 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd9424da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd93c0b5e90 con 0x7fd93c15f260 2024-05-11T01:01:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93b7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd93c0d4ee0 0x7fd93c0d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93b7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd93c0d4ee0 0x7fd93c0d5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47344/0 (socket says 172.21.15.52:47344) 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93b7fe640 1 -- 172.21.15.52:0/3841004500 learned_addr learned my addr 172.21.15.52:0/3841004500 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd940a77640 1 --1- 172.21.15.52:0/3841004500 >> v1:172.21.15.52:6789/0 conn(0x7fd93c0b4e80 0x7fd93c0b5310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51794/0 (socket says 172.21.15.52:51794) 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93b7fe640 1 -- 172.21.15.52:0/3841004500 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 msgr2=0x7fd93c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93bfff640 1 --2- 172.21.15.52:0/3841004500 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 0x7fd93c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93b7fe640 1 --2- 172.21.15.52:0/3841004500 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 0x7fd93c0cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93b7fe640 1 -- 172.21.15.52:0/3841004500 >> v1:172.21.15.52:6789/0 conn(0x7fd93c0b4e80 legacy=0x7fd93c0b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93affd640 1 -- 172.21.15.52:0/3841004500 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377322619 0 0) 0x7fd93c0b5bb0 con 0x7fd93c0b4e80 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93b7fe640 1 -- 172.21.15.52:0/3841004500 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c0d2e00 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.784+0000 7fd93bfff640 1 --2- 172.21.15.52:0/3841004500 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 0x7fd93c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.785+0000 7fd93b7fe640 1 --2- 172.21.15.52:0/3841004500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 0x7fd93c0d5320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd930009870 tx=0x7fd93002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f6dfb09bd750f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.785+0000 7fd93affd640 1 -- 172.21.15.52:0/3841004500 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930037070 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.785+0000 7fd93affd640 1 -- 172.21.15.52:0/3841004500 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd930032440 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.785+0000 7fd93affd640 1 -- 172.21.15.52:0/3841004500 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930032810 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.786+0000 7fd9424da640 1 -- 172.21.15.52:0/3841004500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 msgr2=0x7fd93c0d5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.786+0000 7fd9424da640 1 --2- 172.21.15.52:0/3841004500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 0x7fd93c0d5320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd930009870 tx=0x7fd93002a270 comp rx=0 tx=0).stop 2024-05-11T01:01:39.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.786+0000 7fd9424da640 1 -- 172.21.15.52:0/3841004500 shutdown_connections 2024-05-11T01:01:39.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.786+0000 7fd9424da640 1 --2- 172.21.15.52:0/3841004500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 0x7fd93c0d5320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.786+0000 7fd9424da640 1 --2- 172.21.15.52:0/3841004500 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 0x7fd93c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.786+0000 7fd9424da640 1 -- 172.21.15.52:0/3841004500 >> 172.21.15.52:0/3841004500 conn(0x7fd93c08ebe0 msgr2=0x7fd93c09e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:39.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.786+0000 7fd9424da640 1 -- 172.21.15.52:0/3841004500 shutdown_connections 2024-05-11T01:01:39.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.786+0000 7fd9424da640 1 -- 172.21.15.52:0/3841004500 wait complete. 2024-05-11T01:01:39.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.787+0000 7fd9424da640 1 Processor -- start 2024-05-11T01:01:39.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.787+0000 7fd9424da640 1 -- start start 2024-05-11T01:01:39.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd9424da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd93c0b4e80 0x7fd93c0babf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd9424da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 0x7fd93c0b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd9424da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 0x7fd93c15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd9424da640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c0d0250 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd9424da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c0b6170 con 0x7fd93c0b4e80 2024-05-11T01:01:39.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd9424da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd93c0d03d0 con 0x7fd93c15f260 2024-05-11T01:01:39.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 0x7fd93c0b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 0x7fd93c0b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47348/0 (socket says 172.21.15.52:47348) 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd93b7fe640 1 -- 172.21.15.52:0/995173902 learned_addr learned my addr 172.21.15.52:0/995173902 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd940a77640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 0x7fd93c15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd93b7fe640 1 -- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 msgr2=0x7fd93c15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd93bfff640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd93c0b4e80 0x7fd93c0babf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.788+0000 7fd93b7fe640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 0x7fd93c15d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.789+0000 7fd93b7fe640 1 -- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd93c0b4e80 msgr2=0x7fd93c0babf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.789+0000 7fd93b7fe640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd93c0b4e80 0x7fd93c0babf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.789+0000 7fd93b7fe640 1 -- 172.21.15.52:0/995173902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c0b8c70 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.789+0000 7fd93b7fe640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 0x7fd93c0b8730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd930035040 tx=0x7fd9300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.789+0000 7fd938ff9640 1 -- 172.21.15.52:0/995173902 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930037040 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.789+0000 7fd938ff9640 1 -- 172.21.15.52:0/995173902 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd930004230 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.789+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93c0b8f40 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.789+0000 7fd938ff9640 1 -- 172.21.15.52:0/995173902 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93003c670 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.789+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93c0b9500 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.790+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93c069670 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.791+0000 7fd938ff9640 1 -- 172.21.15.52:0/995173902 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd93003a070 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.791+0000 7fd938ff9640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9280762d0 0x7fd928078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:39.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.791+0000 7fd938ff9640 1 -- 172.21.15.52:0/995173902 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fd9300b79f0 con 0x7fd93c0d4ee0 2024-05-11T01:01:39.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.791+0000 7fd93bfff640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9280762d0 0x7fd928078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:39.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.792+0000 7fd93bfff640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9280762d0 0x7fd928078790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00b840 tx=0x7fd92c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:39.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:39.795+0000 7fd938ff9640 1 -- 172.21.15.52:0/995173902 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd930041020 con 0x7fd93c0d4ee0 2024-05-11T01:01:40.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.058+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fd93c0b4bf0 con 0x7fd93c0d4ee0 2024-05-11T01:01:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.060+0000 7fd938ff9640 1 -- 172.21.15.52:0/995173902 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fd9300816a0 con 0x7fd93c0d4ee0 2024-05-11T01:01:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9280762d0 msgr2=0x7fd928078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9280762d0 0x7fd928078790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00b840 tx=0x7fd92c008040 comp rx=0 tx=0).stop 2024-05-11T01:01:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 msgr2=0x7fd93c0b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 0x7fd93c0b8730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd930035040 tx=0x7fd9300046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 shutdown_connections 2024-05-11T01:01:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9280762d0 0x7fd928078790 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd93c15f260 0x7fd93c15d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd93c0d4ee0 0x7fd93c0b8730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 --2- 172.21.15.52:0/995173902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd93c0b4e80 0x7fd93c0babf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 >> 172.21.15.52:0/995173902 conn(0x7fd93c08ebe0 msgr2=0x7fd93c0b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 shutdown_connections 2024-05-11T01:01:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.063+0000 7fd9424da640 1 -- 172.21.15.52:0/995173902 wait complete. 2024-05-11T01:01:40.074 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-11T01:01:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.178+0000 7f9d3f220640 1 Processor -- start 2024-05-11T01:01:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.178+0000 7f9d3f220640 1 -- start start 2024-05-11T01:01:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.179+0000 7f9d3f220640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9d3815f260 0x7f9d380cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.179+0000 7f9d3f220640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d380b4e80 0x7f9d380b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.179+0000 7f9d3f220640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d380b5bb0 con 0x7f9d380d4ee0 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.179+0000 7f9d3f220640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f9d380d3170 con 0x7f9d3815f260 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.179+0000 7f9d3f220640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9d380b5e90 con 0x7f9d380b4e80 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.179+0000 7f9d3cfbc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9d3815f260 0x7f9d380cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.179+0000 7f9d3cfbc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9d3815f260 0x7f9d380cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47358/0 (socket says 172.21.15.52:47358) 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.179+0000 7f9d37fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f9d380d4ee0 0x7f9d380d5320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51796/0 (socket says 172.21.15.52:51796) 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.179+0000 7f9d3cfbc640 1 -- 172.21.15.52:0/3653760280 learned_addr learned my addr 172.21.15.52:0/3653760280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.180+0000 7f9d3cfbc640 1 -- 172.21.15.52:0/3653760280 >> v2:172.21.15.119:3300/0 conn(0x7f9d380b4e80 msgr2=0x7f9d380b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.180+0000 7f9d3d7bd640 1 --2- 172.21.15.52:0/3653760280 >> v2:172.21.15.119:3300/0 conn(0x7f9d380b4e80 0x7f9d380b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.180+0000 7f9d3cfbc640 1 --2- 172.21.15.52:0/3653760280 >> v2:172.21.15.119:3300/0 conn(0x7f9d380b4e80 0x7f9d380b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.180+0000 7f9d3cfbc640 1 -- 172.21.15.52:0/3653760280 >> v1:172.21.15.52:6789/0 conn(0x7f9d380d4ee0 legacy=0x7f9d380d5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.180+0000 7f9d3cfbc640 1 -- 172.21.15.52:0/3653760280 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d380d2e00 con 0x7f9d3815f260 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.180+0000 7f9d377fe640 1 -- 172.21.15.52:0/3653760280 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 774043235 0 0) 0x7f9d380b5bb0 con 0x7f9d380d4ee0 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.180+0000 7f9d3cfbc640 1 --2- 172.21.15.52:0/3653760280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 0x7f9d380cfd10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009870 tx=0x7f9d2c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50093679d9c5fdd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.180+0000 7f9d377fe640 1 -- 172.21.15.52:0/3653760280 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c037070 con 0x7f9d3815f260 2024-05-11T01:01:40.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d377fe640 1 -- 172.21.15.52:0/3653760280 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2c032440 con 0x7f9d3815f260 2024-05-11T01:01:40.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d377fe640 1 -- 172.21.15.52:0/3653760280 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c032810 con 0x7f9d3815f260 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d3f220640 1 -- 172.21.15.52:0/3653760280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 msgr2=0x7f9d380cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d3f220640 1 --2- 172.21.15.52:0/3653760280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 0x7f9d380cfd10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009870 tx=0x7f9d2c02a2a0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d3f220640 1 -- 172.21.15.52:0/3653760280 shutdown_connections 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d3f220640 1 --2- 172.21.15.52:0/3653760280 >> v2:172.21.15.119:3300/0 conn(0x7f9d380b4e80 0x7f9d380b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d3f220640 1 --2- 172.21.15.52:0/3653760280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 0x7f9d380cfd10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d3f220640 1 -- 172.21.15.52:0/3653760280 >> 172.21.15.52:0/3653760280 conn(0x7f9d3808ebe0 msgr2=0x7f9d3809e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d3f220640 1 -- 172.21.15.52:0/3653760280 shutdown_connections 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.181+0000 7f9d3f220640 1 -- 172.21.15.52:0/3653760280 wait complete. 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.182+0000 7f9d3f220640 1 Processor -- start 2024-05-11T01:01:40.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.182+0000 7f9d3f220640 1 -- start start 2024-05-11T01:01:40.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3f220640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d380b4e80 0x7f9d3815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3f220640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d380d4ee0 0x7f9d380ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3f220640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 0x7f9d380cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3f220640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d380d0250 con 0x7f9d3815f260 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3f220640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d380b5850 con 0x7f9d380d4ee0 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3f220640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f9d380d03d0 con 0x7f9d380b4e80 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3d7bd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 0x7f9d380cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3d7bd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 0x7f9d380cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47362/0 (socket says 172.21.15.52:47362) 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3d7bd640 1 -- 172.21.15.52:0/3892406938 learned_addr learned my addr 172.21.15.52:0/3892406938 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d3cfbc640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d380b4e80 0x7f9d3815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.183+0000 7f9d37fff640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d380d4ee0 0x7f9d380ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d3d7bd640 1 -- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d380b4e80 msgr2=0x7f9d3815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d3d7bd640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d380b4e80 0x7f9d3815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d3d7bd640 1 -- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d380d4ee0 msgr2=0x7f9d380ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d3d7bd640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d380d4ee0 0x7f9d380ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d3d7bd640 1 -- 172.21.15.52:0/3892406938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d380cdfa0 con 0x7f9d3815f260 2024-05-11T01:01:40.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d3d7bd640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 0x7f9d380cda60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000cd70 tx=0x7f9d3000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:40.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d357fa640 1 -- 172.21.15.52:0/3892406938 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30003b90 con 0x7f9d3815f260 2024-05-11T01:01:40.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d357fa640 1 -- 172.21.15.52:0/3892406938 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d30003d30 con 0x7f9d3815f260 2024-05-11T01:01:40.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d380b8770 con 0x7f9d3815f260 2024-05-11T01:01:40.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d357fa640 1 -- 172.21.15.52:0/3892406938 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30004240 con 0x7f9d3815f260 2024-05-11T01:01:40.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.184+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d380b8cf0 con 0x7f9d3815f260 2024-05-11T01:01:40.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.185+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d38069670 con 0x7f9d3815f260 2024-05-11T01:01:40.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.186+0000 7f9d357fa640 1 -- 172.21.15.52:0/3892406938 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9d300043e0 con 0x7f9d3815f260 2024-05-11T01:01:40.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.186+0000 7f9d357fa640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d1c0762d0 0x7f9d1c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.186+0000 7f9d357fa640 1 -- 172.21.15.52:0/3892406938 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f9d30097b70 con 0x7f9d3815f260 2024-05-11T01:01:40.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.186+0000 7f9d3cfbc640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d1c0762d0 0x7f9d1c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.187+0000 7f9d3cfbc640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d1c0762d0 0x7f9d1c078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c034000 tx=0x7f9d2c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:40.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.190+0000 7f9d357fa640 1 -- 172.21.15.52:0/3892406938 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9d380cdfa0 con 0x7f9d3815f260 2024-05-11T01:01:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.457+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f9d3815a950 con 0x7f9d3815f260 2024-05-11T01:01:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.459+0000 7f9d357fa640 1 -- 172.21.15.52:0/3892406938 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f9d30061260 con 0x7f9d3815f260 2024-05-11T01:01:40.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d1c0762d0 msgr2=0x7f9d1c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d1c0762d0 0x7f9d1c078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c034000 tx=0x7f9d2c02e000 comp rx=0 tx=0).stop 2024-05-11T01:01:40.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 msgr2=0x7f9d380cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 0x7f9d380cda60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000cd70 tx=0x7f9d3000a660 comp rx=0 tx=0).stop 2024-05-11T01:01:40.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 shutdown_connections 2024-05-11T01:01:40.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d1c0762d0 0x7f9d1c078790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d3815f260 0x7f9d380cda60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d380d4ee0 0x7f9d380ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 --2- 172.21.15.52:0/3892406938 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d380b4e80 0x7f9d3815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 >> 172.21.15.52:0/3892406938 conn(0x7f9d3808ebe0 msgr2=0x7f9d3806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:40.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 shutdown_connections 2024-05-11T01:01:40.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.461+0000 7f9d3f220640 1 -- 172.21.15.52:0/3892406938 wait complete. 2024-05-11T01:01:40.472 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-11T01:01:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.577+0000 7fadc8f46640 1 Processor -- start 2024-05-11T01:01:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.577+0000 7fadc8f46640 1 -- start start 2024-05-11T01:01:40.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.577+0000 7fadc8f46640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadc40b4e80 0x7fadc40b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.577+0000 7fadc8f46640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40b5bb0 con 0x7fadc40d4ee0 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.577+0000 7fadc8f46640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40b5e90 con 0x7fadc415f260 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.577+0000 7fadc8f46640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fadc40d3170 con 0x7fadc40b4e80 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadbbfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fadc40d4ee0 0x7fadc40d5320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51804/0 (socket says 172.21.15.52:51804) 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadbbfff640 1 -- 172.21.15.52:0/2552059309 learned_addr learned my addr 172.21.15.52:0/2552059309 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc2d9d640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadc40b4e80 0x7fadc40b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 162723795 0 0) 0x7fadc40b5bb0 con 0x7fadc40d4ee0 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadb0003520 con 0x7fadc40d4ee0 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 873624407 0 0) 0x7fadb0003520 con 0x7fadc40d4ee0 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadc40b4e80 msgr2=0x7fadc40b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadc40b4e80 0x7fadc40b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 >> v1:172.21.15.119:6789/0 conn(0x7fadc415f260 legacy=0x7fadc40cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc40d2e00 con 0x7fadc40d4ee0 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fadac0016f0 con 0x7fadc40d4ee0 2024-05-11T01:01:40.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 >> v1:172.21.15.52:6789/0 conn(0x7fadc40d4ee0 legacy=0x7fadc40d5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb00039e0 0x7fadb0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadc40b4e80 0x7fadb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0004d40 0x7fadb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fadc40d3170 con 0x7fadb0004d40 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fadb000baa0 con 0x7fadb00039e0 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fadb000bca0 con 0x7fadc40b4e80 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc2d9d640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0004d40 0x7fadb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadc259c640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb00039e0 0x7fadb0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.578+0000 7fadbbfff640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadc40b4e80 0x7fadb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.579+0000 7fadc2d9d640 1 -- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadc40b4e80 msgr2=0x7fadb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.579+0000 7fadc2d9d640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadc40b4e80 0x7fadb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.579+0000 7fadc2d9d640 1 -- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb00039e0 msgr2=0x7fadb0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.579+0000 7fadc2d9d640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb00039e0 0x7fadb0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.579+0000 7fadc2d9d640 1 -- 172.21.15.52:0/2552059309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb0003520 con 0x7fadb0004d40 2024-05-11T01:01:40.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.579+0000 7fadc2d9d640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0004d40 0x7fadb00051c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009f90 tx=0x7fadb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f34b1d07c39796e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:40.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc009c10 con 0x7fadb0004d40 2024-05-11T01:01:40.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadbc036070 con 0x7fadb0004d40 2024-05-11T01:01:40.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc1d9b640 1 -- 172.21.15.52:0/2552059309 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc031850 con 0x7fadb0004d40 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc8f46640 1 -- 172.21.15.52:0/2552059309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0004d40 msgr2=0x7fadb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc8f46640 1 --2- 172.21.15.52:0/2552059309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0004d40 0x7fadb00051c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009f90 tx=0x7fadb0005830 comp rx=0 tx=0).stop 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc2d9d640 1 -- 172.21.15.52:0/2552059309 reap_dead start 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc8f46640 1 -- 172.21.15.52:0/2552059309 shutdown_connections 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc8f46640 1 -- 172.21.15.52:0/2552059309 >> 172.21.15.52:0/2552059309 conn(0x7fadc408ebe0 msgr2=0x7fadc409e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc8f46640 1 -- 172.21.15.52:0/2552059309 shutdown_connections 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.580+0000 7fadc8f46640 1 -- 172.21.15.52:0/2552059309 wait complete. 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadc8f46640 1 Processor -- start 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadc8f46640 1 -- start start 2024-05-11T01:01:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadc8f46640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadb0004d40 0x7fadc40c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadc8f46640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadc40ca120 0x7fadc41f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadc8f46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc41f5260 0x7fadc41f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadc8f46640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fadc415f7c0 con 0x7fadc40ca120 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadc8f46640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fadc415f640 con 0x7fadc41f5260 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadc8f46640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fadc40b5310 con 0x7fadb0004d40 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadbbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadc40ca120 0x7fadc41f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadbbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadc40ca120 0x7fadc41f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47388/0 (socket says 172.21.15.52:47388) 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.582+0000 7fadbbfff640 1 -- 172.21.15.52:0/2514623682 learned_addr learned my addr 172.21.15.52:0/2514623682 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadbbfff640 1 -- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadb0004d40 msgr2=0x7fadc40c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadbbfff640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadb0004d40 0x7fadc40c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadbbfff640 1 -- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc41f5260 msgr2=0x7fadc41f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadc2d9d640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc41f5260 0x7fadc41f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadbbfff640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc41f5260 0x7fadc41f56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadbbfff640 1 -- 172.21.15.52:0/2514623682 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc41f2700 con 0x7fadc40ca120 2024-05-11T01:01:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadbbfff640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadc40ca120 0x7fadc41f4d20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fadac000fd0 tx=0x7fadac0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadbaffd640 1 -- 172.21.15.52:0/2514623682 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac00f930 con 0x7fadc40ca120 2024-05-11T01:01:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadbaffd640 1 -- 172.21.15.52:0/2514623682 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac018070 con 0x7fadc40ca120 2024-05-11T01:01:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadbaffd640 1 -- 172.21.15.52:0/2514623682 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac00fd30 con 0x7fadc40ca120 2024-05-11T01:01:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc40ca630 con 0x7fadc40ca120 2024-05-11T01:01:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.583+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41f6080 con 0x7fadc40ca120 2024-05-11T01:01:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.584+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc4069670 con 0x7fadc40ca120 2024-05-11T01:01:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.585+0000 7fadbaffd640 1 -- 172.21.15.52:0/2514623682 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fadac01d460 con 0x7fadc40ca120 2024-05-11T01:01:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.585+0000 7fadbaffd640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fada40762d0 0x7fada4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.585+0000 7fadbaffd640 1 -- 172.21.15.52:0/2514623682 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fadac097320 con 0x7fadc40ca120 2024-05-11T01:01:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.586+0000 7fadc259c640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fada40762d0 0x7fada4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.586+0000 7fadc259c640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fada40762d0 0x7fada4078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadc40b3af0 tx=0x7fadb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.590+0000 7fadbaffd640 1 -- 172.21.15.52:0/2514623682 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fadac060fc0 con 0x7fadc40ca120 2024-05-11T01:01:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.857+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fadc40b4bf0 con 0x7fadc40ca120 2024-05-11T01:01:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.859+0000 7fadbaffd640 1 -- 172.21.15.52:0/2514623682 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fadac0608c0 con 0x7fadc40ca120 2024-05-11T01:01:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fada40762d0 msgr2=0x7fada4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fada40762d0 0x7fada4078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadc40b3af0 tx=0x7fadb4009040 comp rx=0 tx=0).stop 2024-05-11T01:01:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadc40ca120 msgr2=0x7fadc41f4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadc40ca120 0x7fadc41f4d20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fadac000fd0 tx=0x7fadac0077a0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 shutdown_connections 2024-05-11T01:01:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fada40762d0 0x7fada4078790 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadc41f5260 0x7fadc41f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadc40ca120 0x7fadc41f4d20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 --2- 172.21.15.52:0/2514623682 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fadb0004d40 0x7fadc40c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 >> 172.21.15.52:0/2514623682 conn(0x7fadc408ebe0 msgr2=0x7fadc40b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 shutdown_connections 2024-05-11T01:01:40.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.861+0000 7fadc8f46640 1 -- 172.21.15.52:0/2514623682 wait complete. 2024-05-11T01:01:40.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-11T01:01:40.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.977+0000 7fdef17af640 1 Processor -- start 2024-05-11T01:01:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.977+0000 7fdef17af640 1 -- start start 2024-05-11T01:01:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.978+0000 7fdef17af640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdeec15d450 0x7fdeec0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.978+0000 7fdef17af640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdeec0d1240 0x7fdeec06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.978+0000 7fdef17af640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.978+0000 7fdef17af640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdeec06ce30 con 0x7fdeec0d1240 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.978+0000 7fdef17af640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fdeec0742c0 con 0x7fdeec15d450 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.978+0000 7fdef17af640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdeec0ce090 con 0x7fdeec06c9c0 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.978+0000 7fdeea7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdeec0d1240 0x7fdeec06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.978+0000 7fdeea7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdeec0d1240 0x7fdeec06c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47394/0 (socket says 172.21.15.52:47394) 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.978+0000 7fdeea7fc640 1 -- 172.21.15.52:0/4170115064 learned_addr learned my addr 172.21.15.52:0/4170115064 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeeb7fe640 1 --2- 172.21.15.52:0/4170115064 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeea7fc640 1 -- 172.21.15.52:0/4170115064 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 msgr2=0x7fdeec0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeea7fc640 1 --2- 172.21.15.52:0/4170115064 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeea7fc640 1 -- 172.21.15.52:0/4170115064 >> v2:172.21.15.119:3300/0 conn(0x7fdeec15d450 msgr2=0x7fdeec0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeeaffd640 1 --2- 172.21.15.52:0/4170115064 >> v2:172.21.15.119:3300/0 conn(0x7fdeec15d450 0x7fdeec0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeea7fc640 1 --2- 172.21.15.52:0/4170115064 >> v2:172.21.15.119:3300/0 conn(0x7fdeec15d450 0x7fdeec0cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeea7fc640 1 -- 172.21.15.52:0/4170115064 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec073d00 con 0x7fdeec0d1240 2024-05-11T01:01:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeeb7fe640 1 --2- 172.21.15.52:0/4170115064 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeeaffd640 1 --2- 172.21.15.52:0/4170115064 >> v2:172.21.15.119:3300/0 conn(0x7fdeec15d450 0x7fdeec0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.979+0000 7fdeea7fc640 1 --2- 172.21.15.52:0/4170115064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec0d1240 0x7fdeec06c480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdeec076850 tx=0x7fded402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdcf3d626a86390 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdee9ffb640 1 -- 172.21.15.52:0/4170115064 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4037070 con 0x7fdeec0d1240 2024-05-11T01:01:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdee9ffb640 1 -- 172.21.15.52:0/4170115064 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded4032440 con 0x7fdeec0d1240 2024-05-11T01:01:40.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdee9ffb640 1 -- 172.21.15.52:0/4170115064 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4032810 con 0x7fdeec0d1240 2024-05-11T01:01:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdef17af640 1 -- 172.21.15.52:0/4170115064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec0d1240 msgr2=0x7fdeec06c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdef17af640 1 --2- 172.21.15.52:0/4170115064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec0d1240 0x7fdeec06c480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdeec076850 tx=0x7fded402a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdef17af640 1 -- 172.21.15.52:0/4170115064 shutdown_connections 2024-05-11T01:01:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdef17af640 1 --2- 172.21.15.52:0/4170115064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec0d1240 0x7fdeec06c480 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdef17af640 1 --2- 172.21.15.52:0/4170115064 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdef17af640 1 --2- 172.21.15.52:0/4170115064 >> v2:172.21.15.119:3300/0 conn(0x7fdeec15d450 0x7fdeec0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdef17af640 1 -- 172.21.15.52:0/4170115064 >> 172.21.15.52:0/4170115064 conn(0x7fdeec0ce9f0 msgr2=0x7fdeec0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdef17af640 1 -- 172.21.15.52:0/4170115064 shutdown_connections 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.980+0000 7fdef17af640 1 -- 172.21.15.52:0/4170115064 wait complete. 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.982+0000 7fdef17af640 1 Processor -- start 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.982+0000 7fdef17af640 1 -- start start 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.982+0000 7fdef17af640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.982+0000 7fdef17af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec15d450 0x7fdeec1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.982+0000 7fdef17af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdeec0c9120 0x7fdeec15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.982+0000 7fdef17af640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec0d1b40 con 0x7fdeec15d450 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.982+0000 7fdef17af640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec0d16d0 con 0x7fdeec0c9120 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.982+0000 7fdef17af640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdeec0d1cc0 con 0x7fdeec06c9c0 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeea7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec15d450 0x7fdeec1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeea7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec15d450 0x7fdeec1542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47408/0 (socket says 172.21.15.52:47408) 2024-05-11T01:01:40.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeea7fc640 1 -- 172.21.15.52:0/2555509356 learned_addr learned my addr 172.21.15.52:0/2555509356 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeeb7fe640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdeec0c9120 0x7fdeec15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeeaffd640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeea7fc640 1 -- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 msgr2=0x7fdeec0c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeea7fc640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeea7fc640 1 -- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdeec0c9120 msgr2=0x7fdeec15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeea7fc640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdeec0c9120 0x7fdeec15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeea7fc640 1 -- 172.21.15.52:0/2555509356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec0ca120 con 0x7fdeec15d450 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeeaffd640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdeea7fc640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec15d450 0x7fdeec1542e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fded402a220 tx=0x7fded40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdee37fe640 1 -- 172.21.15.52:0/2555509356 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4037040 con 0x7fdeec15d450 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdee37fe640 1 -- 172.21.15.52:0/2555509356 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded4042070 con 0x7fdeec15d450 2024-05-11T01:01:40.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeec0ca3f0 con 0x7fdeec15d450 2024-05-11T01:01:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdee37fe640 1 -- 172.21.15.52:0/2555509356 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4030bb0 con 0x7fdeec15d450 2024-05-11T01:01:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.983+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeec0b6310 con 0x7fdeec15d450 2024-05-11T01:01:40.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.984+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeec0b6130 con 0x7fdeec15d450 2024-05-11T01:01:40.987 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:01:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.985+0000 7fdee37fe640 1 -- 172.21.15.52:0/2555509356 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fded4047460 con 0x7fdeec15d450 2024-05-11T01:01:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.986+0000 7fdee37fe640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdecc0762d0 0x7fdecc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.986+0000 7fdee37fe640 1 -- 172.21.15.52:0/2555509356 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fded40b7f00 con 0x7fdeec15d450 2024-05-11T01:01:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.986+0000 7fdeeaffd640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdecc0762d0 0x7fdecc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:40.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.986+0000 7fdeeaffd640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdecc0762d0 0x7fdecc078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00e3d0 tx=0x7fdedc007a10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:40.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:01:40.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:40.990+0000 7fdee37fe640 1 -- 172.21.15.52:0/2555509356 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdeec0b6130 con 0x7fdeec15d450 2024-05-11T01:01:41.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.253+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fdeec069670 con 0x7fdeec15d450 2024-05-11T01:01:41.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.255+0000 7fdee37fe640 1 -- 172.21.15.52:0/2555509356 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fded4081b20 con 0x7fdeec15d450 2024-05-11T01:01:41.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdecc0762d0 msgr2=0x7fdecc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdecc0762d0 0x7fdecc078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00e3d0 tx=0x7fdedc007a10 comp rx=0 tx=0).stop 2024-05-11T01:01:41.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec15d450 msgr2=0x7fdeec1542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec15d450 0x7fdeec1542e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fded402a220 tx=0x7fded40046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 shutdown_connections 2024-05-11T01:01:41.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fdecc0762d0 0x7fdecc078790 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdeec0c9120 0x7fdeec15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdeec15d450 0x7fdeec1542e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 --2- 172.21.15.52:0/2555509356 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdeec06c9c0 0x7fdeec0c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 >> 172.21.15.52:0/2555509356 conn(0x7fdeec0ce9f0 msgr2=0x7fdeec0d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:41.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 shutdown_connections 2024-05-11T01:01:41.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.257+0000 7fdef17af640 1 -- 172.21.15.52:0/2555509356 wait complete. 2024-05-11T01:01:41.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-11T01:01:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.373+0000 7f00abc5c640 1 Processor -- start 2024-05-11T01:01:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.373+0000 7f00abc5c640 1 -- start start 2024-05-11T01:01:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.374+0000 7f00abc5c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f00a40d4ee0 0x7f00a40d5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.374+0000 7f00abc5c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00a40b4e80 0x7f00a40b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.374+0000 7f00abc5c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a40b5bb0 con 0x7f00a415f260 2024-05-11T01:01:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.374+0000 7f00abc5c640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f00a40d3170 con 0x7f00a40d4ee0 2024-05-11T01:01:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.374+0000 7f00abc5c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f00a40b5e90 con 0x7f00a40b4e80 2024-05-11T01:01:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.374+0000 7f00a91f7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f00a40d4ee0 0x7f00a40d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.374+0000 7f00a91f7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f00a40d4ee0 0x7f00a40d5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47420/0 (socket says 172.21.15.52:47420) 2024-05-11T01:01:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.374+0000 7f00a91f7640 1 -- 172.21.15.52:0/520117049 learned_addr learned my addr 172.21.15.52:0/520117049 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.374+0000 7f00aa1f9640 1 --2- 172.21.15.52:0/520117049 >> v2:172.21.15.119:3300/0 conn(0x7f00a40b4e80 0x7f00a40b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.375+0000 7f00a91f7640 1 -- 172.21.15.52:0/520117049 >> v2:172.21.15.119:3300/0 conn(0x7f00a40b4e80 msgr2=0x7f00a40b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.375+0000 7f00a91f7640 1 --2- 172.21.15.52:0/520117049 >> v2:172.21.15.119:3300/0 conn(0x7f00a40b4e80 0x7f00a40b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.375+0000 7f00a91f7640 1 -- 172.21.15.52:0/520117049 >> v1:172.21.15.119:6789/0 conn(0x7f00a415f260 legacy=0x7f00a40cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.375+0000 7f00a91f7640 1 -- 172.21.15.52:0/520117049 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a40d2e00 con 0x7f00a40d4ee0 2024-05-11T01:01:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.375+0000 7f00aa1f9640 1 --2- 172.21.15.52:0/520117049 >> v2:172.21.15.119:3300/0 conn(0x7f00a40b4e80 0x7f00a40b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.375+0000 7f00a91f7640 1 --2- 172.21.15.52:0/520117049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 0x7f00a40d5320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0094009870 tx=0x7f009402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1134edc38ef81850 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:41.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.375+0000 7f00a89f6640 1 -- 172.21.15.52:0/520117049 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094037070 con 0x7f00a40d4ee0 2024-05-11T01:01:41.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.375+0000 7f00a89f6640 1 -- 172.21.15.52:0/520117049 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0094032440 con 0x7f00a40d4ee0 2024-05-11T01:01:41.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.375+0000 7f00a89f6640 1 -- 172.21.15.52:0/520117049 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094032810 con 0x7f00a40d4ee0 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.376+0000 7f00abc5c640 1 -- 172.21.15.52:0/520117049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 msgr2=0x7f00a40d5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.376+0000 7f00abc5c640 1 --2- 172.21.15.52:0/520117049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 0x7f00a40d5320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0094009870 tx=0x7f009402a2a0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.376+0000 7f00abc5c640 1 -- 172.21.15.52:0/520117049 shutdown_connections 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.376+0000 7f00abc5c640 1 --2- 172.21.15.52:0/520117049 >> v2:172.21.15.119:3300/0 conn(0x7f00a40b4e80 0x7f00a40b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.376+0000 7f00abc5c640 1 --2- 172.21.15.52:0/520117049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 0x7f00a40d5320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.376+0000 7f00abc5c640 1 -- 172.21.15.52:0/520117049 >> 172.21.15.52:0/520117049 conn(0x7f00a408ebe0 msgr2=0x7f00a409e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.376+0000 7f00abc5c640 1 -- 172.21.15.52:0/520117049 shutdown_connections 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.376+0000 7f00abc5c640 1 -- 172.21.15.52:0/520117049 wait complete. 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.377+0000 7f00abc5c640 1 Processor -- start 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.377+0000 7f00abc5c640 1 -- start start 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.378+0000 7f00abc5c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00a40b4e80 0x7f00a4157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.378+0000 7f00abc5c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 0x7f00a4161e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.378+0000 7f00abc5c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00a415f260 0x7f00a4162360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.378+0000 7f00abc5c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f00a40b5850 con 0x7f00a40d4ee0 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.378+0000 7f00abc5c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00a40b6170 con 0x7f00a415f260 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.378+0000 7f00abc5c640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f00a415f640 con 0x7f00a40b4e80 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.378+0000 7f00a91f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 0x7f00a4161e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.378+0000 7f00a91f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 0x7f00a4161e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47428/0 (socket says 172.21.15.52:47428) 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.378+0000 7f00a91f7640 1 -- 172.21.15.52:0/1359721290 learned_addr learned my addr 172.21.15.52:0/1359721290 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00aa1f9640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00a415f260 0x7f00a4162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00a99f8640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00a40b4e80 0x7f00a4157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00a91f7640 1 -- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00a40b4e80 msgr2=0x7f00a4157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00a91f7640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00a40b4e80 0x7f00a4157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00a91f7640 1 -- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00a415f260 msgr2=0x7f00a4162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00a91f7640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00a415f260 0x7f00a4162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00a91f7640 1 -- 172.21.15.52:0/1359721290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a4157740 con 0x7f00a40d4ee0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00a91f7640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 0x7f00a4161e20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0094000c00 tx=0x7f00940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f009a7fc640 1 -- 172.21.15.52:0/1359721290 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094041020 con 0x7f00a40d4ee0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f009a7fc640 1 -- 172.21.15.52:0/1359721290 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00940306c0 con 0x7f00a40d4ee0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f009a7fc640 1 -- 172.21.15.52:0/1359721290 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094037040 con 0x7f00a40d4ee0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a41607f0 con 0x7f00a40d4ee0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.379+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a4160db0 con 0x7f00a40d4ee0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.380+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a40ccfc0 con 0x7f00a40d4ee0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.381+0000 7f009a7fc640 1 -- 172.21.15.52:0/1359721290 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0094044020 con 0x7f00a40d4ee0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.381+0000 7f009a7fc640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f00880762d0 0x7f0088078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.381+0000 7f009a7fc640 1 -- 172.21.15.52:0/1359721290 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f00940b76f0 con 0x7f00a40d4ee0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.382+0000 7f00a99f8640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f00880762d0 0x7f0088078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.382+0000 7f00a99f8640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f00880762d0 0x7f0088078790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f00a00015e0 tx=0x7f00a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:41.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.386+0000 7f009a7fc640 1 -- 172.21.15.52:0/1359721290 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f00a40ccfc0 con 0x7f00a40d4ee0 2024-05-11T01:01:41.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.651+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f00a4069670 con 0x7f00a40d4ee0 2024-05-11T01:01:41.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.653+0000 7f009a7fc640 1 -- 172.21.15.52:0/1359721290 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f009404b070 con 0x7f00a40d4ee0 2024-05-11T01:01:41.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f00880762d0 msgr2=0x7f0088078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f00880762d0 0x7f0088078790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f00a00015e0 tx=0x7f00a0009040 comp rx=0 tx=0).stop 2024-05-11T01:01:41.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 msgr2=0x7f00a4161e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 0x7f00a4161e20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0094000c00 tx=0x7f00940046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 shutdown_connections 2024-05-11T01:01:41.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f00880762d0 0x7f0088078790 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00a415f260 0x7f00a4162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00a40d4ee0 0x7f00a4161e20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 --2- 172.21.15.52:0/1359721290 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00a40b4e80 0x7f00a4157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.655+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 >> 172.21.15.52:0/1359721290 conn(0x7f00a408ebe0 msgr2=0x7f00a40d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:41.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.656+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 shutdown_connections 2024-05-11T01:01:41.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.656+0000 7f00abc5c640 1 -- 172.21.15.52:0/1359721290 wait complete. 2024-05-11T01:01:41.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-11T01:01:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.777+0000 7f71e930a640 1 Processor -- start 2024-05-11T01:01:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.777+0000 7f71e930a640 1 -- start start 2024-05-11T01:01:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.778+0000 7f71e930a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e406c480 0x7f71e40aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.778+0000 7f71e930a640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e40b4880 con 0x7f71e409fc70 2024-05-11T01:01:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.778+0000 7f71e930a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e4073380 con 0x7f71e4164c50 2024-05-11T01:01:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.778+0000 7f71e930a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f71e4070290 con 0x7f71e406c480 2024-05-11T01:01:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.778+0000 7f71e27fc640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f71e409fc70 0x7f71e406c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33800/0 (socket says 172.21.15.52:33800) 2024-05-11T01:01:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.778+0000 7f71e27fc640 1 -- 172.21.15.52:0/3120839838 learned_addr learned my addr 172.21.15.52:0/3120839838 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:41.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.779+0000 7f71e2ffd640 1 --2- 172.21.15.52:0/3120839838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e406c480 0x7f71e40aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.779+0000 7f71e1ffb640 1 -- 172.21.15.52:0/3120839838 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3973619860 0 0) 0x7f71e40b4880 con 0x7f71e409fc70 2024-05-11T01:01:41.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.779+0000 7f71e1ffb640 1 -- 172.21.15.52:0/3120839838 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71d0003520 con 0x7f71e409fc70 2024-05-11T01:01:41.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.779+0000 7f71e1ffb640 1 -- 172.21.15.52:0/3120839838 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034975465 0 0) 0x7f71e4073380 con 0x7f71e4164c50 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.779+0000 7f71e1ffb640 1 -- 172.21.15.52:0/3120839838 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71e40b4880 con 0x7f71e4164c50 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.779+0000 7f71e2ffd640 1 -- 172.21.15.52:0/3120839838 >> v1:172.21.15.119:6789/0 conn(0x7f71e4164c50 legacy=0x7f71e4167040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.779+0000 7f71e2ffd640 1 -- 172.21.15.52:0/3120839838 >> v1:172.21.15.52:6789/0 conn(0x7f71e409fc70 legacy=0x7f71e406c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.779+0000 7f71e1ffb640 1 -- 172.21.15.52:0/3120839838 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3883870037 0 0) 0x7f71d0003520 con 0x7f71e409fc70 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.779+0000 7f71e2ffd640 1 -- 172.21.15.52:0/3120839838 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e4073660 con 0x7f71e406c480 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.780+0000 7f71e2ffd640 1 --2- 172.21.15.52:0/3120839838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e406c480 0x7f71e40aedd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009dc0 tx=0x7f71dc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17444ab8ac30369 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.780+0000 7f71e1ffb640 1 -- 172.21.15.52:0/3120839838 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0016f0 con 0x7f71e406c480 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.780+0000 7f71e1ffb640 1 -- 172.21.15.52:0/3120839838 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc007540 con 0x7f71e406c480 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.780+0000 7f71e1ffb640 1 -- 172.21.15.52:0/3120839838 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0078f0 con 0x7f71e406c480 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.780+0000 7f71e930a640 1 -- 172.21.15.52:0/3120839838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e406c480 msgr2=0x7f71e40aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.780+0000 7f71e930a640 1 --2- 172.21.15.52:0/3120839838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e406c480 0x7f71e40aedd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009dc0 tx=0x7f71dc02a660 comp rx=0 tx=0).stop 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.781+0000 7f71e930a640 1 -- 172.21.15.52:0/3120839838 shutdown_connections 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.781+0000 7f71e930a640 1 --2- 172.21.15.52:0/3120839838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e406c480 0x7f71e40aedd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.781+0000 7f71e930a640 1 -- 172.21.15.52:0/3120839838 >> 172.21.15.52:0/3120839838 conn(0x7f71e40b3af0 msgr2=0x7f71e40b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:41.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.781+0000 7f71e930a640 1 -- 172.21.15.52:0/3120839838 shutdown_connections 2024-05-11T01:01:41.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.781+0000 7f71e930a640 1 -- 172.21.15.52:0/3120839838 wait complete. 2024-05-11T01:01:41.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.782+0000 7f71e930a640 1 Processor -- start 2024-05-11T01:01:41.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.782+0000 7f71e930a640 1 -- start start 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e930a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71e406c480 0x7f71e4146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e930a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e409fc70 0x7f71e4143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e930a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e4164c50 0x7f71e409f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e930a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4074140 con 0x7f71e4164c50 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e930a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4073c20 con 0x7f71e406c480 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e930a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f71e40af4b0 con 0x7f71e409fc70 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e4164c50 0x7f71e409f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e4164c50 0x7f71e409f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40660/0 (socket says 172.21.15.52:40660) 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e37fe640 1 -- 172.21.15.52:0/2902323777 learned_addr learned my addr 172.21.15.52:0/2902323777 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e2ffd640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71e406c480 0x7f71e4146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e27fc640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e409fc70 0x7f71e4143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e37fe640 1 -- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e409fc70 msgr2=0x7f71e4143350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e37fe640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e409fc70 0x7f71e4143350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e37fe640 1 -- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71e406c480 msgr2=0x7f71e4146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e37fe640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71e406c480 0x7f71e4146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e37fe640 1 -- 172.21.15.52:0/2902323777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e4144390 con 0x7f71e4164c50 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e37fe640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e4164c50 0x7f71e409f070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f71d400c3d0 tx=0x7f71d400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71db7fe640 1 -- 172.21.15.52:0/2902323777 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4017070 con 0x7f71e4164c50 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71db7fe640 1 -- 172.21.15.52:0/2902323777 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d40052f0 con 0x7f71e4164c50 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e4143af0 con 0x7f71e4164c50 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.783+0000 7f71db7fe640 1 -- 172.21.15.52:0/2902323777 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4003830 con 0x7f71e4164c50 2024-05-11T01:01:41.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.784+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e41440b0 con 0x7f71e4164c50 2024-05-11T01:01:41.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.785+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e4069670 con 0x7f71e4164c50 2024-05-11T01:01:41.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.785+0000 7f71db7fe640 1 -- 172.21.15.52:0/2902323777 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f71d401a070 con 0x7f71e4164c50 2024-05-11T01:01:41.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.786+0000 7f71db7fe640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71c40762d0 0x7f71c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:41.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.786+0000 7f71db7fe640 1 -- 172.21.15.52:0/2902323777 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f71d4096610 con 0x7f71e4164c50 2024-05-11T01:01:41.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.786+0000 7f71e2ffd640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71c40762d0 0x7f71c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:41.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.786+0000 7f71e2ffd640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71c40762d0 0x7f71c4078790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f71dc034000 tx=0x7f71dc02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:41.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:41.790+0000 7f71db7fe640 1 -- 172.21.15.52:0/2902323777 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f71d40602c0 con 0x7f71e4164c50 2024-05-11T01:01:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.054+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f71e41605c0 con 0x7f71e4164c50 2024-05-11T01:01:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.056+0000 7f71db7fe640 1 -- 172.21.15.52:0/2902323777 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f71d40682b0 con 0x7f71e4164c50 2024-05-11T01:01:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71c40762d0 msgr2=0x7f71c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71c40762d0 0x7f71c4078790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f71dc034000 tx=0x7f71dc02f000 comp rx=0 tx=0).stop 2024-05-11T01:01:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e4164c50 msgr2=0x7f71e409f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e4164c50 0x7f71e409f070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f71d400c3d0 tx=0x7f71d400c890 comp rx=0 tx=0).stop 2024-05-11T01:01:42.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 shutdown_connections 2024-05-11T01:01:42.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f71c40762d0 0x7f71c4078790 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e4164c50 0x7f71e409f070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f71e409fc70 0x7f71e4143350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 --2- 172.21.15.52:0/2902323777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71e406c480 0x7f71e4146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 >> 172.21.15.52:0/2902323777 conn(0x7f71e40b3af0 msgr2=0x7f71e40d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:42.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 shutdown_connections 2024-05-11T01:01:42.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.058+0000 7f71e930a640 1 -- 172.21.15.52:0/2902323777 wait complete. 2024-05-11T01:01:42.070 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T01:01:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.175+0000 7f5ec439d640 1 Processor -- start 2024-05-11T01:01:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.175+0000 7f5ec439d640 1 -- start start 2024-05-11T01:01:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec439d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc15f630 0x7f5ebc0d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec439d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ebc0d4f50 0x7f5ebc0d53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec439d640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc0d0540 con 0x7f5ebc163ac0 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec439d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc15fa10 con 0x7f5ebc0d4f50 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec439d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc0d1150 con 0x7f5ebc15f630 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec1938640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f5ebc163ac0 0x7f5ebc0cfd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33810/0 (socket says 172.21.15.52:33810) 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec1938640 1 -- 172.21.15.52:0/1563812052 learned_addr learned my addr 172.21.15.52:0/1563812052 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec2139640 1 --2- 172.21.15.52:0/1563812052 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc15f630 0x7f5ebc0d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec293a640 1 --2- 172.21.15.52:0/1563812052 >> v2:172.21.15.119:3300/0 conn(0x7f5ebc0d4f50 0x7f5ebc0d53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec1137640 1 -- 172.21.15.52:0/1563812052 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2391675228 0 0) 0x7f5ebc0d0540 con 0x7f5ebc163ac0 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec1137640 1 -- 172.21.15.52:0/1563812052 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ea8003520 con 0x7f5ebc163ac0 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec2139640 1 -- 172.21.15.52:0/1563812052 >> v2:172.21.15.119:3300/0 conn(0x7f5ebc0d4f50 msgr2=0x7f5ebc0d53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec2139640 1 --2- 172.21.15.52:0/1563812052 >> v2:172.21.15.119:3300/0 conn(0x7f5ebc0d4f50 0x7f5ebc0d53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec2139640 1 -- 172.21.15.52:0/1563812052 >> v1:172.21.15.52:6789/0 conn(0x7f5ebc163ac0 legacy=0x7f5ebc0cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec2139640 1 -- 172.21.15.52:0/1563812052 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0d0820 con 0x7f5ebc15f630 2024-05-11T01:01:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.176+0000 7f5ec293a640 1 --2- 172.21.15.52:0/1563812052 >> v2:172.21.15.119:3300/0 conn(0x7f5ebc0d4f50 0x7f5ebc0d53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.177+0000 7f5ec2139640 1 --2- 172.21.15.52:0/1563812052 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc15f630 0x7f5ebc0d15e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009d90 tx=0x7f5eb802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987b8ef7cde2a3a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.177+0000 7f5ec1137640 1 -- 172.21.15.52:0/1563812052 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8007740 con 0x7f5ebc15f630 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.177+0000 7f5ec1137640 1 -- 172.21.15.52:0/1563812052 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb80078e0 con 0x7f5ebc15f630 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.177+0000 7f5ec1137640 1 -- 172.21.15.52:0/1563812052 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8007c90 con 0x7f5ebc15f630 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.178+0000 7f5ec439d640 1 -- 172.21.15.52:0/1563812052 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc15f630 msgr2=0x7f5ebc0d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.178+0000 7f5ec439d640 1 --2- 172.21.15.52:0/1563812052 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc15f630 0x7f5ebc0d15e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009d90 tx=0x7f5eb802a660 comp rx=0 tx=0).stop 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.178+0000 7f5ec439d640 1 -- 172.21.15.52:0/1563812052 shutdown_connections 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.178+0000 7f5ec439d640 1 --2- 172.21.15.52:0/1563812052 >> v2:172.21.15.119:3300/0 conn(0x7f5ebc0d4f50 0x7f5ebc0d53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.178+0000 7f5ec439d640 1 --2- 172.21.15.52:0/1563812052 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc15f630 0x7f5ebc0d15e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.178+0000 7f5ec439d640 1 -- 172.21.15.52:0/1563812052 >> 172.21.15.52:0/1563812052 conn(0x7f5ebc08cfb0 msgr2=0x7f5ebc09e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.178+0000 7f5ec439d640 1 -- 172.21.15.52:0/1563812052 shutdown_connections 2024-05-11T01:01:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.178+0000 7f5ec439d640 1 -- 172.21.15.52:0/1563812052 wait complete. 2024-05-11T01:01:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.179+0000 7f5ec439d640 1 Processor -- start 2024-05-11T01:01:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.179+0000 7f5ec439d640 1 -- start start 2024-05-11T01:01:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec439d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ebc0d4f50 0x7f5ebc15cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec439d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ebc15f630 0x7f5ebc160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec439d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc163ac0 0x7f5ebc0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec439d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0d5920 con 0x7f5ebc15f630 2024-05-11T01:01:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec439d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0d0de0 con 0x7f5ebc0d4f50 2024-05-11T01:01:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec439d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc163f20 con 0x7f5ebc163ac0 2024-05-11T01:01:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec1938640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ebc15f630 0x7f5ebc160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec1938640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ebc15f630 0x7f5ebc160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40668/0 (socket says 172.21.15.52:40668) 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec1938640 1 -- 172.21.15.52:0/1820814526 learned_addr learned my addr 172.21.15.52:0/1820814526 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec293a640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc163ac0 0x7f5ebc0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec2139640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ebc0d4f50 0x7f5ebc15cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec1938640 1 -- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc163ac0 msgr2=0x7f5ebc0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec1938640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc163ac0 0x7f5ebc0cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec1938640 1 -- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ebc0d4f50 msgr2=0x7f5ebc15cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec1938640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ebc0d4f50 0x7f5ebc15cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec1938640 1 -- 172.21.15.52:0/1820814526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0cc830 con 0x7f5ebc15f630 2024-05-11T01:01:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.180+0000 7f5ec293a640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc163ac0 0x7f5ebc0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.181+0000 7f5ec1938640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ebc15f630 0x7f5ebc160630 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00c500 tx=0x7f5eac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.181+0000 7f5eb2ffd640 1 -- 172.21.15.52:0/1820814526 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac017070 con 0x7f5ebc15f630 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.181+0000 7f5eb2ffd640 1 -- 172.21.15.52:0/1820814526 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac005ce0 con 0x7f5ebc15f630 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.181+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc160d40 con 0x7f5ebc15f630 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.181+0000 7f5eb2ffd640 1 -- 172.21.15.52:0/1820814526 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac007270 con 0x7f5ebc15f630 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.181+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc161300 con 0x7f5ebc15f630 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.182+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc0696a0 con 0x7f5ebc15f630 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.182+0000 7f5eb2ffd640 1 -- 172.21.15.52:0/1820814526 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5eac01a070 con 0x7f5ebc15f630 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.183+0000 7f5eb2ffd640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ea00762d0 0x7f5ea0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.183+0000 7f5eb2ffd640 1 -- 172.21.15.52:0/1820814526 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f5eac096500 con 0x7f5ebc15f630 2024-05-11T01:01:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.183+0000 7f5ec2139640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ea00762d0 0x7f5ea0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.184+0000 7f5ec2139640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ea00762d0 0x7f5ea0078790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8034000 tx=0x7f5eb802f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.187+0000 7f5eb2ffd640 1 -- 172.21.15.52:0/1820814526 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5eac05fbf0 con 0x7f5ebc15f630 2024-05-11T01:01:42.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.380+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5ebc0b5870 con 0x7f5ebc15f630 2024-05-11T01:01:42.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.380+0000 7f5eb2ffd640 1 -- 172.21.15.52:0/1820814526 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) ==== 84+0+65 (secure 0 0 0) 0x7f5eac063ef0 con 0x7f5ebc15f630 2024-05-11T01:01:42.380 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:42.380 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-05-11T01:01:42.380 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:42.381 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-05-11T01:01:42.381 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-05-11T01:01:42.381 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:42.381 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ea00762d0 msgr2=0x7f5ea0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ea00762d0 0x7f5ea0078790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8034000 tx=0x7f5eb802f000 comp rx=0 tx=0).stop 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ebc15f630 msgr2=0x7f5ebc160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ebc15f630 0x7f5ebc160630 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00c500 tx=0x7f5eac00c9c0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 shutdown_connections 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f5ea00762d0 0x7f5ea0078790 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ebc163ac0 0x7f5ebc0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ebc15f630 0x7f5ebc160630 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 --2- 172.21.15.52:0/1820814526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ebc0d4f50 0x7f5ebc15cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 >> 172.21.15.52:0/1820814526 conn(0x7f5ebc08cfb0 msgr2=0x7f5ebc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 shutdown_connections 2024-05-11T01:01:42.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.382+0000 7f5ec439d640 1 -- 172.21.15.52:0/1820814526 wait complete. 2024-05-11T01:01:42.393 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T01:01:42.393 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T01:01:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.501+0000 7f09e4c45640 1 Processor -- start 2024-05-11T01:01:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.501+0000 7f09e4c45640 1 -- start start 2024-05-11T01:01:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09e4c45640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09e015f310 0x7f09e00d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09e4c45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e00b4dc0 0x7f09e00d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09e4c45640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e00cfd40 con 0x7f09e00d55d0 2024-05-11T01:01:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09e4c45640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f09e00d5a60 con 0x7f09e015f310 2024-05-11T01:01:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09e4c45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f09e00b6140 con 0x7f09e00b4dc0 2024-05-11T01:01:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09ded9d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f09e00d55d0 0x7f09e0162ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33814/0 (socket says 172.21.15.52:33814) 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09ded9d640 1 -- 172.21.15.52:0/642032874 learned_addr learned my addr 172.21.15.52:0/642032874 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09de59c640 1 --2- 172.21.15.52:0/642032874 >> v2:172.21.15.119:3300/0 conn(0x7f09e015f310 0x7f09e00d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09ddd9b640 1 --2- 172.21.15.52:0/642032874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e00b4dc0 0x7f09e00d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.502+0000 7f09dd59a640 1 -- 172.21.15.52:0/642032874 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3045356730 0 0) 0x7f09d8001110 con 0x7f09e00d55d0 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.503+0000 7f09dd59a640 1 -- 172.21.15.52:0/642032874 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09cc003520 con 0x7f09e00d55d0 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.503+0000 7f09dd59a640 1 -- 172.21.15.52:0/642032874 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1219200711 0 0) 0x7f09cc003520 con 0x7f09e00d55d0 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.503+0000 7f09ddd9b640 1 -- 172.21.15.52:0/642032874 >> v2:172.21.15.119:3300/0 conn(0x7f09e015f310 msgr2=0x7f09e00d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.503+0000 7f09ddd9b640 1 --2- 172.21.15.52:0/642032874 >> v2:172.21.15.119:3300/0 conn(0x7f09e015f310 0x7f09e00d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.503+0000 7f09ddd9b640 1 -- 172.21.15.52:0/642032874 >> v1:172.21.15.52:6789/0 conn(0x7f09e00d55d0 legacy=0x7f09e0162ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.503+0000 7f09ddd9b640 1 -- 172.21.15.52:0/642032874 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e00b5530 con 0x7f09e00b4dc0 2024-05-11T01:01:42.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.503+0000 7f09de59c640 1 --2- 172.21.15.52:0/642032874 >> v2:172.21.15.119:3300/0 conn(0x7f09e015f310 0x7f09e00d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.504+0000 7f09ddd9b640 1 --2- 172.21.15.52:0/642032874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e00b4dc0 0x7f09e00d5090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f09c80097a0 tx=0x7f09c802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b10f1609e7ed78 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.504+0000 7f09dd59a640 1 -- 172.21.15.52:0/642032874 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8037070 con 0x7f09e00b4dc0 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.504+0000 7f09dd59a640 1 -- 172.21.15.52:0/642032874 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c802aba0 con 0x7f09e00b4dc0 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.504+0000 7f09dd59a640 1 -- 172.21.15.52:0/642032874 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80018c0 con 0x7f09e00b4dc0 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.504+0000 7f09e4c45640 1 -- 172.21.15.52:0/642032874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e00b4dc0 msgr2=0x7f09e00d5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.504+0000 7f09e4c45640 1 --2- 172.21.15.52:0/642032874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e00b4dc0 0x7f09e00d5090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f09c80097a0 tx=0x7f09c802a110 comp rx=0 tx=0).stop 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.504+0000 7f09e4c45640 1 -- 172.21.15.52:0/642032874 shutdown_connections 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.504+0000 7f09e4c45640 1 --2- 172.21.15.52:0/642032874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e00b4dc0 0x7f09e00d5090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.504+0000 7f09e4c45640 1 --2- 172.21.15.52:0/642032874 >> v2:172.21.15.119:3300/0 conn(0x7f09e015f310 0x7f09e00d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.505+0000 7f09e4c45640 1 -- 172.21.15.52:0/642032874 >> 172.21.15.52:0/642032874 conn(0x7f09e008eb80 msgr2=0x7f09e009e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:42.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.505+0000 7f09e4c45640 1 -- 172.21.15.52:0/642032874 shutdown_connections 2024-05-11T01:01:42.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.505+0000 7f09e4c45640 1 -- 172.21.15.52:0/642032874 wait complete. 2024-05-11T01:01:42.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.506+0000 7f09e4c45640 1 Processor -- start 2024-05-11T01:01:42.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.506+0000 7f09e4c45640 1 -- start start 2024-05-11T01:01:42.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09e4c45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e00b4dc0 0x7f09e01503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09e4c45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09e00d55d0 0x7f09e00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09e4c45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e015f310 0x7f09e00cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09e4c45640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09e00b5dd0 con 0x7f09e00b4dc0 2024-05-11T01:01:42.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09e4c45640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09e00b5af0 con 0x7f09e00d55d0 2024-05-11T01:01:42.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09e4c45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f09e00b5250 con 0x7f09e015f310 2024-05-11T01:01:42.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09de59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e00b4dc0 0x7f09e01503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09de59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e00b4dc0 0x7f09e01503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40680/0 (socket says 172.21.15.52:40680) 2024-05-11T01:01:42.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09de59c640 1 -- 172.21.15.52:0/3856244954 learned_addr learned my addr 172.21.15.52:0/3856244954 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09ded9d640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e015f310 0x7f09e00cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09ddd9b640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09e00d55d0 0x7f09e00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09de59c640 1 -- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e015f310 msgr2=0x7f09e00cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09de59c640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e015f310 0x7f09e00cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09de59c640 1 -- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09e00d55d0 msgr2=0x7f09e00cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09de59c640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09e00d55d0 0x7f09e00cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09de59c640 1 -- 172.21.15.52:0/3856244954 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e0150900 con 0x7f09e00b4dc0 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09de59c640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e00b4dc0 0x7f09e01503c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09d000efc0 tx=0x7f09d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.507+0000 7f09d6ffd640 1 -- 172.21.15.52:0/3856244954 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0018020 con 0x7f09e00b4dc0 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.508+0000 7f09d6ffd640 1 -- 172.21.15.52:0/3856244954 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d0014680 con 0x7f09e00b4dc0 2024-05-11T01:01:42.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.508+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e015a630 con 0x7f09e00b4dc0 2024-05-11T01:01:42.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.508+0000 7f09d6ffd640 1 -- 172.21.15.52:0/3856244954 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0014a50 con 0x7f09e00b4dc0 2024-05-11T01:01:42.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.508+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e015fbf0 con 0x7f09e00b4dc0 2024-05-11T01:01:42.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.509+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e0069780 con 0x7f09e00b4dc0 2024-05-11T01:01:42.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.509+0000 7f09d6ffd640 1 -- 172.21.15.52:0/3856244954 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f09d0005d10 con 0x7f09e00b4dc0 2024-05-11T01:01:42.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.510+0000 7f09d6ffd640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f09c00762d0 0x7f09c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:42.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.510+0000 7f09d6ffd640 1 -- 172.21.15.52:0/3856244954 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f09d0097f50 con 0x7f09e00b4dc0 2024-05-11T01:01:42.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.510+0000 7f09ddd9b640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f09c00762d0 0x7f09c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:42.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.511+0000 7f09ddd9b640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f09c00762d0 0x7f09c0078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f09c8034000 tx=0x7f09c802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:42.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.515+0000 7f09d6ffd640 1 -- 172.21.15.52:0/3856244954 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f09d0022030 con 0x7f09e00b4dc0 2024-05-11T01:01:42.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:42.706+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f09e00b45c0 con 0x7f09e00b4dc0 2024-05-11T01:01:43.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.262+0000 7f09d6ffd640 1 -- 172.21.15.52:0/3856244954 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) ==== 138+0+0 (secure 0 0 0) 0x7f09d0010040 con 0x7f09e00b4dc0 2024-05-11T01:01:43.262 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-05-11T01:01:43.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f09c00762d0 msgr2=0x7f09c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:43.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f09c00762d0 0x7f09c0078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f09c8034000 tx=0x7f09c802e000 comp rx=0 tx=0).stop 2024-05-11T01:01:43.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e00b4dc0 msgr2=0x7f09e01503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:43.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e00b4dc0 0x7f09e01503c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09d000efc0 tx=0x7f09d000c490 comp rx=0 tx=0).stop 2024-05-11T01:01:43.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 shutdown_connections 2024-05-11T01:01:43.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f09c00762d0 0x7f09c0078790 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09e015f310 0x7f09e00cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09e00d55d0 0x7f09e00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 --2- 172.21.15.52:0/3856244954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09e00b4dc0 0x7f09e01503c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 >> 172.21.15.52:0/3856244954 conn(0x7f09e008eb80 msgr2=0x7f09e015d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:43.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 shutdown_connections 2024-05-11T01:01:43.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.266+0000 7f09e4c45640 1 -- 172.21.15.52:0/3856244954 wait complete. 2024-05-11T01:01:43.284 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T01:01:43.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.401+0000 7fa2dd5b6640 1 Processor -- start 2024-05-11T01:01:43.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.401+0000 7fa2dd5b6640 1 -- start start 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2dd5b6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2d81609b0 0x7fa2d80ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2dd5b6640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d8068a50 0x7fa2d80b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2dd5b6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa2d80b0660 0x7fa2d80b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2dd5b6640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d80b3850 con 0x7fa2d80b0660 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2dd5b6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d80cb7f0 con 0x7fa2d81609b0 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2dd5b6640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d80cbd20 con 0x7fa2d8068a50 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2d77fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa2d80b0660 0x7fa2d80b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2d77fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa2d80b0660 0x7fa2d80b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40688/0 (socket says 172.21.15.52:40688) 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2d77fe640 1 -- 172.21.15.52:0/2438620383 learned_addr learned my addr 172.21.15.52:0/2438620383 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2ce7fc640 1 --2- 172.21.15.52:0/2438620383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d8068a50 0x7fa2d80b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:43.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2d6ffd640 1 --2- 172.21.15.52:0/2438620383 >> v2:172.21.15.119:3300/0 conn(0x7fa2d81609b0 0x7fa2d80ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:43.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2d77fe640 1 -- 172.21.15.52:0/2438620383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d8068a50 msgr2=0x7fa2d80b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:43.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2d77fe640 1 --2- 172.21.15.52:0/2438620383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d8068a50 0x7fa2d80b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2d77fe640 1 -- 172.21.15.52:0/2438620383 >> v2:172.21.15.119:3300/0 conn(0x7fa2d81609b0 msgr2=0x7fa2d80ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:43.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2d77fe640 1 --2- 172.21.15.52:0/2438620383 >> v2:172.21.15.119:3300/0 conn(0x7fa2d81609b0 0x7fa2d80ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.402+0000 7fa2d77fe640 1 -- 172.21.15.52:0/2438620383 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d80b3aa0 con 0x7fa2d80b0660 2024-05-11T01:01:43.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.403+0000 7fa2d77fe640 1 --2- 172.21.15.52:0/2438620383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 0x7fa2d80b10e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8071690 tx=0x7fa2d002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d604077fed1d389 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:43.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.403+0000 7fa2d67fc640 1 -- 172.21.15.52:0/2438620383 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0036070 con 0x7fa2d80b0660 2024-05-11T01:01:43.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.403+0000 7fa2d67fc640 1 -- 172.21.15.52:0/2438620383 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d0031600 con 0x7fa2d80b0660 2024-05-11T01:01:43.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.403+0000 7fa2d67fc640 1 -- 172.21.15.52:0/2438620383 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00319d0 con 0x7fa2d80b0660 2024-05-11T01:01:43.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.404+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/2438620383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 msgr2=0x7fa2d80b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:43.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.404+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/2438620383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 0x7fa2d80b10e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8071690 tx=0x7fa2d002a6a0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.404+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/2438620383 shutdown_connections 2024-05-11T01:01:43.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.404+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/2438620383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 0x7fa2d80b10e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.404+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/2438620383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d8068a50 0x7fa2d80b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.404+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/2438620383 >> v2:172.21.15.119:3300/0 conn(0x7fa2d81609b0 0x7fa2d80ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.404+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/2438620383 >> 172.21.15.52:0/2438620383 conn(0x7fa2d80b2d30 msgr2=0x7fa2d80b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:43.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.404+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/2438620383 shutdown_connections 2024-05-11T01:01:43.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.404+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/2438620383 wait complete. 2024-05-11T01:01:43.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.405+0000 7fa2dd5b6640 1 Processor -- start 2024-05-11T01:01:43.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.405+0000 7fa2dd5b6640 1 -- start start 2024-05-11T01:01:43.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2dd5b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8068a50 0x7fa2d80cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:43.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2dd5b6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 0x7fa2d80b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:43.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2dd5b6640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d81609b0 0x7fa2d81f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:43.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2dd5b6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d80b2a30 con 0x7fa2d80b0660 2024-05-11T01:01:43.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2dd5b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8160d90 con 0x7fa2d8068a50 2024-05-11T01:01:43.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2dd5b6640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d80b0110 con 0x7fa2d81609b0 2024-05-11T01:01:43.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2ce7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 0x7fa2d80b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:43.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2ce7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 0x7fa2d80b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40698/0 (socket says 172.21.15.52:40698) 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2ce7fc640 1 -- 172.21.15.52:0/1973221965 learned_addr learned my addr 172.21.15.52:0/1973221965 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2d77fe640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d81609b0 0x7fa2d81f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2d6ffd640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8068a50 0x7fa2d80cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2ce7fc640 1 -- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d81609b0 msgr2=0x7fa2d81f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.406+0000 7fa2ce7fc640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d81609b0 0x7fa2d81f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.407+0000 7fa2ce7fc640 1 -- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8068a50 msgr2=0x7fa2d80cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.407+0000 7fa2ce7fc640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8068a50 0x7fa2d80cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.407+0000 7fa2ce7fc640 1 -- 172.21.15.52:0/1973221965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d815a3b0 con 0x7fa2d80b0660 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.407+0000 7fa2ce7fc640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 0x7fa2d80b3ce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000e890 tx=0x7fa2c000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.407+0000 7fa2cffff640 1 -- 172.21.15.52:0/1973221965 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0017070 con 0x7fa2d80b0660 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.407+0000 7fa2cffff640 1 -- 172.21.15.52:0/1973221965 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c000c940 con 0x7fa2d80b0660 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.407+0000 7fa2cffff640 1 -- 172.21.15.52:0/1973221965 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000cd10 con 0x7fa2d80b0660 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.407+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d80b6680 con 0x7fa2d80b0660 2024-05-11T01:01:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.407+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d80b6c40 con 0x7fa2d80b0660 2024-05-11T01:01:43.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.408+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d80b6e40 con 0x7fa2d80b0660 2024-05-11T01:01:43.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.409+0000 7fa2cffff640 1 -- 172.21.15.52:0/1973221965 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa2c001a070 con 0x7fa2d80b0660 2024-05-11T01:01:43.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.409+0000 7fa2cffff640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa2b8076300 0x7fa2b80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:43.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.409+0000 7fa2cffff640 1 -- 172.21.15.52:0/1973221965 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5227+0+0 (secure 0 0 0) 0x7fa2c0097360 con 0x7fa2d80b0660 2024-05-11T01:01:43.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.410+0000 7fa2d6ffd640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa2b8076300 0x7fa2b80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:43.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.410+0000 7fa2d6ffd640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa2b8076300 0x7fa2b80787c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009a30 tx=0x7fa2c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:43.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.414+0000 7fa2cffff640 1 -- 172.21.15.52:0/1973221965 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa2c0060ea0 con 0x7fa2d80b0660 2024-05-11T01:01:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:43.608+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa2d815afc0 con 0x7fa2d80b0660 2024-05-11T01:01:44.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.268+0000 7fa2cffff640 1 -- 172.21.15.52:0/1973221965 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) ==== 170+0+0 (secure 0 0 0) 0x7fa2c00607a0 con 0x7fa2d80b0660 2024-05-11T01:01:44.268 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-05-11T01:01:44.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.272+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa2b8076300 msgr2=0x7fa2b80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:44.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.272+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa2b8076300 0x7fa2b80787c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009a30 tx=0x7fa2c8008040 comp rx=0 tx=0).stop 2024-05-11T01:01:44.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.272+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 msgr2=0x7fa2d80b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:44.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.272+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 0x7fa2d80b3ce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000e890 tx=0x7fa2c000ed50 comp rx=0 tx=0).stop 2024-05-11T01:01:44.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.273+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 shutdown_connections 2024-05-11T01:01:44.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.273+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fa2b8076300 0x7fa2b80787c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:44.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.273+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa2d81609b0 0x7fa2d81f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:44.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.273+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2d80b0660 0x7fa2d80b3ce0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:44.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.273+0000 7fa2dd5b6640 1 --2- 172.21.15.52:0/1973221965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2d8068a50 0x7fa2d80cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:44.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.273+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 >> 172.21.15.52:0/1973221965 conn(0x7fa2d80b2d30 msgr2=0x7fa2d80cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:44.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.274+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 shutdown_connections 2024-05-11T01:01:44.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.275+0000 7fa2dd5b6640 1 -- 172.21.15.52:0/1973221965 wait complete. 2024-05-11T01:01:44.302 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T01:01:44.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.414+0000 7fc216fdd640 1 Processor -- start 2024-05-11T01:01:44.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.414+0000 7fc216fdd640 1 -- start start 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc216fdd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc2100b4320 0x7fc21006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc216fdd640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc21006e140 con 0x7fc2100d2b40 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc216fdd640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc21006e420 con 0x7fc21009c130 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc216fdd640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc21006dd80 con 0x7fc2100b4320 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc214d79640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc2100b4320 0x7fc21006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc20ffff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc2100d2b40 0x7fc21009ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33818/0 (socket says 172.21.15.52:33818) 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc20ffff640 1 -- 172.21.15.52:0/654721917 learned_addr learned my addr 172.21.15.52:0/654721917 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc214d79640 1 --2- 172.21.15.52:0/654721917 >> v2:172.21.15.52:3300/0 conn(0x7fc2100b4320 0x7fc21006d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40708/0 (socket says 172.21.15.52:40708) 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc214d79640 1 -- 172.21.15.52:0/654721917 >> v1:172.21.15.119:6789/0 conn(0x7fc21009c130 legacy=0x7fc2100b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc214d79640 1 -- 172.21.15.52:0/654721917 >> v1:172.21.15.52:6789/0 conn(0x7fc2100d2b40 legacy=0x7fc21009ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:44.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc214d79640 1 -- 172.21.15.52:0/654721917 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2100d1e60 con 0x7fc2100b4320 2024-05-11T01:01:44.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.415+0000 7fc20f7fe640 1 -- 172.21.15.52:0/654721917 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 42648670 0 0) 0x7fc21006e140 con 0x7fc2100d2b40 2024-05-11T01:01:44.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.416+0000 7fc214d79640 1 --2- 172.21.15.52:0/654721917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 0x7fc21006d7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc2040097a0 tx=0x7fc20402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30821d399a91942d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.416+0000 7fc20f7fe640 1 -- 172.21.15.52:0/654721917 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204037070 con 0x7fc2100b4320 2024-05-11T01:01:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.416+0000 7fc20f7fe640 1 -- 172.21.15.52:0/654721917 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc20402ae50 con 0x7fc2100b4320 2024-05-11T01:01:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.416+0000 7fc20f7fe640 1 -- 172.21.15.52:0/654721917 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204032660 con 0x7fc2100b4320 2024-05-11T01:01:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.417+0000 7fc216fdd640 1 -- 172.21.15.52:0/654721917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 msgr2=0x7fc21006d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.417+0000 7fc216fdd640 1 --2- 172.21.15.52:0/654721917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 0x7fc21006d7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc2040097a0 tx=0x7fc20402a100 comp rx=0 tx=0).stop 2024-05-11T01:01:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.417+0000 7fc216fdd640 1 -- 172.21.15.52:0/654721917 shutdown_connections 2024-05-11T01:01:44.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.417+0000 7fc216fdd640 1 --2- 172.21.15.52:0/654721917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 0x7fc21006d7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.417+0000 7fc216fdd640 1 -- 172.21.15.52:0/654721917 >> 172.21.15.52:0/654721917 conn(0x7fc2100b4a60 msgr2=0x7fc2100b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.417+0000 7fc216fdd640 1 -- 172.21.15.52:0/654721917 shutdown_connections 2024-05-11T01:01:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.417+0000 7fc216fdd640 1 -- 172.21.15.52:0/654721917 wait complete. 2024-05-11T01:01:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.418+0000 7fc216fdd640 1 Processor -- start 2024-05-11T01:01:44.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.418+0000 7fc216fdd640 1 -- start start 2024-05-11T01:01:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc216fdd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21009c130 0x7fc2101f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:44.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc216fdd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 0x7fc21015a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc216fdd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2100d2b40 0x7fc21015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc216fdd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2100b5980 con 0x7fc2100b4320 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc216fdd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2100b5800 con 0x7fc21009c130 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc216fdd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc2100d2380 con 0x7fc2100d2b40 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc20ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 0x7fc21015a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc20ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 0x7fc21015a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40724/0 (socket says 172.21.15.52:40724) 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc20ffff640 1 -- 172.21.15.52:0/3295417231 learned_addr learned my addr 172.21.15.52:0/3295417231 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc214d79640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21009c130 0x7fc2101f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc21557a640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2100d2b40 0x7fc21015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc20ffff640 1 -- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2100d2b40 msgr2=0x7fc21015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc20ffff640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2100d2b40 0x7fc21015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc20ffff640 1 -- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21009c130 msgr2=0x7fc2101f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc20ffff640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21009c130 0x7fc2101f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.419+0000 7fc20ffff640 1 -- 172.21.15.52:0/3295417231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2100c7ca0 con 0x7fc2100b4320 2024-05-11T01:01:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.420+0000 7fc20ffff640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 0x7fc21015a940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c410 tx=0x7fc1fc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.420+0000 7fc20d7fa640 1 -- 172.21.15.52:0/3295417231 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc0074e0 con 0x7fc2100b4320 2024-05-11T01:01:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.420+0000 7fc20d7fa640 1 -- 172.21.15.52:0/3295417231 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1fc0047e0 con 0x7fc2100b4320 2024-05-11T01:01:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.420+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2100c7f80 con 0x7fc2100b4320 2024-05-11T01:01:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.420+0000 7fc20d7fa640 1 -- 172.21.15.52:0/3295417231 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc004bd0 con 0x7fc2100b4320 2024-05-11T01:01:44.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.420+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2100c8540 con 0x7fc2100b4320 2024-05-11T01:01:44.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.421+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2100695d0 con 0x7fc2100b4320 2024-05-11T01:01:44.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.422+0000 7fc20d7fa640 1 -- 172.21.15.52:0/3295417231 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc1fc011070 con 0x7fc2100b4320 2024-05-11T01:01:44.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.422+0000 7fc20d7fa640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc1f40762d0 0x7fc1f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:44.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.422+0000 7fc20d7fa640 1 -- 172.21.15.52:0/3295417231 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5609+0+0 (secure 0 0 0) 0x7fc1fc097130 con 0x7fc2100b4320 2024-05-11T01:01:44.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.422+0000 7fc214d79640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc1f40762d0 0x7fc1f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:44.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.423+0000 7fc214d79640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc1f40762d0 0x7fc1f4078790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc204034000 tx=0x7fc20402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:44.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.427+0000 7fc20d7fa640 1 -- 172.21.15.52:0/3295417231 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc1fc060b80 con 0x7fc2100b4320 2024-05-11T01:01:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:44.659+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc2100b8ef0 con 0x7fc2100b4320 2024-05-11T01:01:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.269+0000 7fc20d7fa640 1 -- 172.21.15.52:0/3295417231 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fc1fc060480 con 0x7fc2100b4320 2024-05-11T01:01:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 4 and data pool 5 2024-05-11T01:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.288+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc1f40762d0 msgr2=0x7fc1f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.288+0000 7fc216fdd640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc1f40762d0 0x7fc1f4078790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc204034000 tx=0x7fc20402e000 comp rx=0 tx=0).stop 2024-05-11T01:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.288+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 msgr2=0x7fc21015a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.288+0000 7fc216fdd640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 0x7fc21015a940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c410 tx=0x7fc1fc00c8d0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.292+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 shutdown_connections 2024-05-11T01:01:45.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.292+0000 7fc216fdd640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fc1f40762d0 0x7fc1f4078790 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.292+0000 7fc216fdd640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc2100d2b40 0x7fc21015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.292+0000 7fc216fdd640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2100b4320 0x7fc21015a940 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.292+0000 7fc216fdd640 1 --2- 172.21.15.52:0/3295417231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc21009c130 0x7fc2101f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.292+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 >> 172.21.15.52:0/3295417231 conn(0x7fc2100b4a60 msgr2=0x7fc21015e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:45.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.292+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 shutdown_connections 2024-05-11T01:01:45.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.293+0000 7fc216fdd640 1 -- 172.21.15.52:0/3295417231 wait complete. 2024-05-11T01:01:45.314 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.422+0000 7effdcc45640 1 Processor -- start 2024-05-11T01:01:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.422+0000 7effdcc45640 1 -- start start 2024-05-11T01:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effdcc45640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7effd815f430 0x7effd815fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effdcc45640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7effd81602b0 0x7effd80d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effdcc45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 0x7effd80d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effdcc45640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7effd8163250 con 0x7effd815f430 2024-05-11T01:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effdcc45640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7effd8163530 con 0x7effd81602b0 2024-05-11T01:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effdcc45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7effd8163810 con 0x7effd8160720 2024-05-11T01:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effd659c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7effd815f430 0x7effd815fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effd659c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7effd815f430 0x7effd815fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40734/0 (socket says 172.21.15.52:40734) 2024-05-11T01:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effd659c640 1 -- 172.21.15.52:0/146609180 learned_addr learned my addr 172.21.15.52:0/146609180 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effd659c640 1 -- 172.21.15.52:0/146609180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 msgr2=0x7effd80d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effd659c640 1 --2- 172.21.15.52:0/146609180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 0x7effd80d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effd659c640 1 -- 172.21.15.52:0/146609180 >> v2:172.21.15.119:3300/0 conn(0x7effd81602b0 msgr2=0x7effd80d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effd659c640 1 --2- 172.21.15.52:0/146609180 >> v2:172.21.15.119:3300/0 conn(0x7effd81602b0 0x7effd80d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.423+0000 7effd659c640 1 -- 172.21.15.52:0/146609180 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd8163af0 con 0x7effd815f430 2024-05-11T01:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.424+0000 7effd659c640 1 --2- 172.21.15.52:0/146609180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd815f430 0x7effd815fd70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7effd80b4840 tx=0x7effc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee49a1088edbadb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.424+0000 7effd5d9b640 1 -- 172.21.15.52:0/146609180 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8037070 con 0x7effd815f430 2024-05-11T01:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.424+0000 7effd5d9b640 1 -- 172.21.15.52:0/146609180 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc8032440 con 0x7effd815f430 2024-05-11T01:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.424+0000 7effd5d9b640 1 -- 172.21.15.52:0/146609180 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8032810 con 0x7effd815f430 2024-05-11T01:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.424+0000 7effdcc45640 1 -- 172.21.15.52:0/146609180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd815f430 msgr2=0x7effd815fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.424+0000 7effdcc45640 1 --2- 172.21.15.52:0/146609180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd815f430 0x7effd815fd70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7effd80b4840 tx=0x7effc802a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.425+0000 7effdcc45640 1 -- 172.21.15.52:0/146609180 shutdown_connections 2024-05-11T01:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.425+0000 7effdcc45640 1 --2- 172.21.15.52:0/146609180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 0x7effd80d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.425+0000 7effdcc45640 1 --2- 172.21.15.52:0/146609180 >> v2:172.21.15.119:3300/0 conn(0x7effd81602b0 0x7effd80d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.425+0000 7effdcc45640 1 --2- 172.21.15.52:0/146609180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd815f430 0x7effd815fd70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.425+0000 7effdcc45640 1 -- 172.21.15.52:0/146609180 >> 172.21.15.52:0/146609180 conn(0x7effd808eb70 msgr2=0x7effd809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.425+0000 7effdcc45640 1 -- 172.21.15.52:0/146609180 shutdown_connections 2024-05-11T01:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.425+0000 7effdcc45640 1 -- 172.21.15.52:0/146609180 wait complete. 2024-05-11T01:01:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.426+0000 7effdcc45640 1 Processor -- start 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.426+0000 7effdcc45640 1 -- start start 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effdcc45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effd815f430 0x7effd80ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effdcc45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd81602b0 0x7effd81e92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effdcc45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 0x7effd80d0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effdcc45640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7effd8163030 con 0x7effd81602b0 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effdcc45640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7effd8162eb0 con 0x7effd815f430 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effdcc45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7effd80d25d0 con 0x7effd8160720 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effcdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd81602b0 0x7effd81e92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effcdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd81602b0 0x7effd81e92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40750/0 (socket says 172.21.15.52:40750) 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effcdd9b640 1 -- 172.21.15.52:0/2456779852 learned_addr learned my addr 172.21.15.52:0/2456779852 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effd659c640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effd815f430 0x7effd80ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effd6d9d640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 0x7effd80d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effcdd9b640 1 -- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 msgr2=0x7effd80d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effcdd9b640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 0x7effd80d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effcdd9b640 1 -- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effd815f430 msgr2=0x7effd80ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effcdd9b640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effd815f430 0x7effd80ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effcdd9b640 1 -- 172.21.15.52:0/2456779852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd814efe0 con 0x7effd81602b0 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effd6d9d640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 0x7effd80d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.427+0000 7effcdd9b640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd81602b0 0x7effd81e92e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7effc000b4a0 tx=0x7effc000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.428+0000 7effcf7fe640 1 -- 172.21.15.52:0/2456779852 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0014070 con 0x7effd81602b0 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.428+0000 7effcf7fe640 1 -- 172.21.15.52:0/2456779852 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc0010610 con 0x7effd81602b0 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.428+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd814f2c0 con 0x7effd81602b0 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.428+0000 7effcf7fe640 1 -- 172.21.15.52:0/2456779852 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0010a00 con 0x7effd81602b0 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.428+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd814f880 con 0x7effd81602b0 2024-05-11T01:01:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.429+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd814fa80 con 0x7effd81602b0 2024-05-11T01:01:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.429+0000 7effcf7fe640 1 -- 172.21.15.52:0/2456779852 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7effc0017070 con 0x7effd81602b0 2024-05-11T01:01:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.430+0000 7effcf7fe640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7effb80762d0 0x7effb8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.430+0000 7effcf7fe640 1 -- 172.21.15.52:0/2456779852 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7effc0096f50 con 0x7effd81602b0 2024-05-11T01:01:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.430+0000 7effd659c640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7effb80762d0 0x7effb8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.431+0000 7effd659c640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7effb80762d0 0x7effb8078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effc8034000 tx=0x7effc802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:45.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.434+0000 7effcf7fe640 1 -- 172.21.15.52:0/2456779852 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7effd814fa80 con 0x7effd81602b0 2024-05-11T01:01:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.627+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effd8069780 con 0x7effd81602b0 2024-05-11T01:01:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.628+0000 7effcf7fe640 1 -- 172.21.15.52:0/2456779852 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16682 (secure 0 0 0) 0x7effc00602f0 con 0x7effd81602b0 2024-05-11T01:01:45.628 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:45.628 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:44.660536+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:01:42.710844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:43.612754+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7effb80762d0 msgr2=0x7effb8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7effb80762d0 0x7effb8078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effc8034000 tx=0x7effc802e000 comp rx=0 tx=0).stop 2024-05-11T01:01:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd81602b0 msgr2=0x7effd81e92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd81602b0 0x7effd81e92e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7effc000b4a0 tx=0x7effc000b960 comp rx=0 tx=0).stop 2024-05-11T01:01:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 shutdown_connections 2024-05-11T01:01:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7effb80762d0 0x7effb8078790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effd8160720 0x7effd80d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effd81602b0 0x7effd81e92e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 --2- 172.21.15.52:0/2456779852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effd815f430 0x7effd80ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 >> 172.21.15.52:0/2456779852 conn(0x7effd808eb70 msgr2=0x7effd806dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 shutdown_connections 2024-05-11T01:01:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.630+0000 7effdcc45640 1 -- 172.21.15.52:0/2456779852 wait complete. 2024-05-11T01:01:45.642 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:45.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.745+0000 7f9d57a59640 1 Processor -- start 2024-05-11T01:01:45.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.745+0000 7f9d57a59640 1 -- start start 2024-05-11T01:01:45.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.746+0000 7f9d57a59640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d501576a0 0x7f9d500add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.746+0000 7f9d57a59640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d500ae290 0x7f9d500af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.746+0000 7f9d57a59640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d500743e0 con 0x7f9d5015e1d0 2024-05-11T01:01:45.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.746+0000 7f9d57a59640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9d50160c50 con 0x7f9d501576a0 2024-05-11T01:01:45.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.746+0000 7f9d57a59640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f9d50160f30 con 0x7f9d500ae290 2024-05-11T01:01:45.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d54ff4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d500ae290 0x7f9d500af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d557f5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d501576a0 0x7f9d500add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d54ff4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d500ae290 0x7f9d500af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:33780/0 (socket says 172.21.15.52:33780) 2024-05-11T01:01:45.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d54ff4640 1 -- 172.21.15.52:0/1508927431 learned_addr learned my addr 172.21.15.52:0/1508927431 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:45.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d557f5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d501576a0 0x7f9d500add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:53192/0 (socket says 172.21.15.52:53192) 2024-05-11T01:01:45.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d55ff6640 1 --1- 172.21.15.52:0/1508927431 >> v1:172.21.15.119:6789/0 conn(0x7f9d5015e1d0 0x7f9d500721b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:38724/0 (socket says 172.21.15.52:38724) 2024-05-11T01:01:45.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d54ff4640 1 -- 172.21.15.52:0/1508927431 >> v2:172.21.15.119:3300/0 conn(0x7f9d501576a0 msgr2=0x7f9d500add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:45.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d54ff4640 1 --2- 172.21.15.52:0/1508927431 >> v2:172.21.15.119:3300/0 conn(0x7f9d501576a0 0x7f9d500add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d54ff4640 1 -- 172.21.15.52:0/1508927431 >> v1:172.21.15.119:6789/0 conn(0x7f9d5015e1d0 legacy=0x7f9d500721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d54ff4640 1 -- 172.21.15.52:0/1508927431 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d50161210 con 0x7f9d500ae290 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d47fff640 1 -- 172.21.15.52:0/1508927431 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3027787873 0 0) 0x7f9d500743e0 con 0x7f9d5015e1d0 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.747+0000 7f9d557f5640 1 --2- 172.21.15.52:0/1508927431 >> v2:172.21.15.119:3300/0 conn(0x7f9d501576a0 0x7f9d500add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.748+0000 7f9d54ff4640 1 --2- 172.21.15.52:0/1508927431 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d500ae290 0x7f9d500af950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009870 tx=0x7f9d4002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94308218a89a8821 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.748+0000 7f9d47fff640 1 -- 172.21.15.52:0/1508927431 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40037070 con 0x7f9d500ae290 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.748+0000 7f9d47fff640 1 -- 172.21.15.52:0/1508927431 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d40032440 con 0x7f9d500ae290 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.748+0000 7f9d47fff640 1 -- 172.21.15.52:0/1508927431 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40032810 con 0x7f9d500ae290 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.749+0000 7f9d57a59640 1 -- 172.21.15.52:0/1508927431 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d500ae290 msgr2=0x7f9d500af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.749+0000 7f9d57a59640 1 --2- 172.21.15.52:0/1508927431 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d500ae290 0x7f9d500af950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009870 tx=0x7f9d4002a270 comp rx=0 tx=0).stop 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.749+0000 7f9d57a59640 1 -- 172.21.15.52:0/1508927431 shutdown_connections 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.749+0000 7f9d57a59640 1 --2- 172.21.15.52:0/1508927431 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d500ae290 0x7f9d500af950 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.749+0000 7f9d57a59640 1 --2- 172.21.15.52:0/1508927431 >> v2:172.21.15.119:3300/0 conn(0x7f9d501576a0 0x7f9d500add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.749+0000 7f9d57a59640 1 -- 172.21.15.52:0/1508927431 >> 172.21.15.52:0/1508927431 conn(0x7f9d500b2a00 msgr2=0x7f9d500b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:45.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.749+0000 7f9d57a59640 1 -- 172.21.15.52:0/1508927431 shutdown_connections 2024-05-11T01:01:45.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.749+0000 7f9d57a59640 1 -- 172.21.15.52:0/1508927431 wait complete. 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.750+0000 7f9d57a59640 1 Processor -- start 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.750+0000 7f9d57a59640 1 -- start start 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d57a59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d500ae290 0x7f9d5014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d57a59640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d501576a0 0x7f9d50143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d57a59640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d5015e1d0 0x7f9d501f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d57a59640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d500afe90 con 0x7f9d501576a0 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d57a59640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5006e050 con 0x7f9d500ae290 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d57a59640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f9d500b0010 con 0x7f9d5015e1d0 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d54ff4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d501576a0 0x7f9d50143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d557f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d500ae290 0x7f9d5014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d54ff4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d501576a0 0x7f9d50143800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40756/0 (socket says 172.21.15.52:40756) 2024-05-11T01:01:45.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d54ff4640 1 -- 172.21.15.52:0/4110441429 learned_addr learned my addr 172.21.15.52:0/4110441429 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d55ff6640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d5015e1d0 0x7f9d501f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d54ff4640 1 -- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d5015e1d0 msgr2=0x7f9d501f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d54ff4640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d5015e1d0 0x7f9d501f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d54ff4640 1 -- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d500ae290 msgr2=0x7f9d5014fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d54ff4640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d500ae290 0x7f9d5014fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d54ff4640 1 -- 172.21.15.52:0/4110441429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d501f8fc0 con 0x7f9d501576a0 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d557f5640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d500ae290 0x7f9d5014fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.751+0000 7f9d54ff4640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d501576a0 0x7f9d50143800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d400090b0 tx=0x7f9d400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.752+0000 7f9d45ffb640 1 -- 172.21.15.52:0/4110441429 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40037040 con 0x7f9d501576a0 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.752+0000 7f9d45ffb640 1 -- 172.21.15.52:0/4110441429 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d40004030 con 0x7f9d501576a0 2024-05-11T01:01:45.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.752+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d501f92a0 con 0x7f9d501576a0 2024-05-11T01:01:45.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.752+0000 7f9d45ffb640 1 -- 172.21.15.52:0/4110441429 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40004420 con 0x7f9d501576a0 2024-05-11T01:01:45.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.752+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d50151840 con 0x7f9d501576a0 2024-05-11T01:01:45.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.753+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d50069780 con 0x7f9d501576a0 2024-05-11T01:01:45.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.754+0000 7f9d45ffb640 1 -- 172.21.15.52:0/4110441429 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9d4003a070 con 0x7f9d501576a0 2024-05-11T01:01:45.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.754+0000 7f9d45ffb640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d340762d0 0x7f9d34078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:45.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.754+0000 7f9d45ffb640 1 -- 172.21.15.52:0/4110441429 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f9d400b7aa0 con 0x7f9d501576a0 2024-05-11T01:01:45.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.755+0000 7f9d557f5640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d340762d0 0x7f9d34078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:45.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.755+0000 7f9d557f5640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d340762d0 0x7f9d34078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009560 tx=0x7f9d4c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:45.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.758+0000 7f9d45ffb640 1 -- 172.21.15.52:0/4110441429 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9d40041020 con 0x7f9d501576a0 2024-05-11T01:01:45.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.951+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d500b30c0 con 0x7f9d501576a0 2024-05-11T01:01:45.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.953+0000 7f9d45ffb640 1 -- 172.21.15.52:0/4110441429 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16682 (secure 0 0 0) 0x7f9d400893f0 con 0x7f9d501576a0 2024-05-11T01:01:45.953 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:45.953 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:44.660536+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:01:42.710844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:43.612754+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d340762d0 msgr2=0x7f9d34078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d340762d0 0x7f9d34078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009560 tx=0x7f9d4c009190 comp rx=0 tx=0).stop 2024-05-11T01:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d501576a0 msgr2=0x7f9d50143800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d501576a0 0x7f9d50143800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d400090b0 tx=0x7f9d400046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 shutdown_connections 2024-05-11T01:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9d340762d0 0x7f9d34078790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d5015e1d0 0x7f9d501f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d501576a0 0x7f9d50143800 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 --2- 172.21.15.52:0/4110441429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d500ae290 0x7f9d5014fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 >> 172.21.15.52:0/4110441429 conn(0x7f9d500b2a00 msgr2=0x7f9d500b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 shutdown_connections 2024-05-11T01:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:45.955+0000 7f9d57a59640 1 -- 172.21.15.52:0/4110441429 wait complete. 2024-05-11T01:01:45.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T01:01:46.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.073+0000 7fd4a7c2a640 1 Processor -- start 2024-05-11T01:01:46.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.073+0000 7fd4a7c2a640 1 -- start start 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a7c2a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4a015f430 0x7fd4a00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a7c2a640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a00cfc60 con 0x7fd4a00d0700 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a7c2a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a0163120 con 0x7fd4a00b5420 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a7c2a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd4a00d2a80 con 0x7fd4a015f430 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a61c7640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd4a00d0700 0x7fd4a00d4e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33834/0 (socket says 172.21.15.52:33834) 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a61c7640 1 -- 172.21.15.52:0/2494510158 learned_addr learned my addr 172.21.15.52:0/2494510158 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a59c6640 1 --2- 172.21.15.52:0/2494510158 >> v2:172.21.15.119:3300/0 conn(0x7fd4a015f430 0x7fd4a00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 459969779 0 0) 0x7fd4a00cfc60 con 0x7fd4a00d0700 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd48c003520 con 0x7fd4a00d0700 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3002381468 0 0) 0x7fd48c003520 con 0x7fd4a00d0700 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 >> v2:172.21.15.119:3300/0 conn(0x7fd4a015f430 msgr2=0x7fd4a00b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 --2- 172.21.15.52:0/2494510158 >> v2:172.21.15.119:3300/0 conn(0x7fd4a015f430 0x7fd4a00b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 >> v1:172.21.15.119:6789/0 conn(0x7fd4a00b5420 legacy=0x7fd4a00d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a00d24c0 con 0x7fd4a00d0700 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fd498001a70 con 0x7fd4a00d0700 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 >> v1:172.21.15.52:6789/0 conn(0x7fd4a00d0700 legacy=0x7fd4a00d4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c003a50 0x7fd48c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd48c004800 0x7fd48c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48c0051c0 0x7fd48c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a00d2a80 con 0x7fd48c003a50 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c00c160 con 0x7fd48c0051c0 2024-05-11T01:01:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.074+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c00c360 con 0x7fd48c004800 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a59c6640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c003a50 0x7fd48c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a61c7640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48c0051c0 0x7fd48c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a51c5640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd48c004800 0x7fd48c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a59c6640 1 -- 172.21.15.52:0/2494510158 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd48c004800 msgr2=0x7fd48c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a59c6640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd48c004800 0x7fd48c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a59c6640 1 -- 172.21.15.52:0/2494510158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48c0051c0 msgr2=0x7fd48c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a59c6640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48c0051c0 0x7fd48c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a59c6640 1 -- 172.21.15.52:0/2494510158 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c003520 con 0x7fd48c003a50 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a61c7640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48c0051c0 0x7fd48c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.075+0000 7fd4a61c7640 1 -- 172.21.15.52:0/2494510158 reap_dead start 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a59c6640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c003a50 0x7fd48c003e70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009ac0 tx=0x7fd48c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651ebdfbf61c8ed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c036070 con 0x7fd48c003a50 2024-05-11T01:01:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd49c031400 con 0x7fd48c003a50 2024-05-11T01:01:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a49c4640 1 -- 172.21.15.52:0/2494510158 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c009690 con 0x7fd48c003a50 2024-05-11T01:01:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/2494510158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c003a50 msgr2=0x7fd48c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a7c2a640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c003a50 0x7fd48c003e70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009ac0 tx=0x7fd48c005830 comp rx=0 tx=0).stop 2024-05-11T01:01:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/2494510158 shutdown_connections 2024-05-11T01:01:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a7c2a640 1 --2- 172.21.15.52:0/2494510158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c003a50 0x7fd48c003e70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/2494510158 >> 172.21.15.52:0/2494510158 conn(0x7fd4a008ebb0 msgr2=0x7fd4a009e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/2494510158 shutdown_connections 2024-05-11T01:01:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.076+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/2494510158 wait complete. 2024-05-11T01:01:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.077+0000 7fd4a7c2a640 1 Processor -- start 2024-05-11T01:01:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a7c2a640 1 -- start start 2024-05-11T01:01:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a7c2a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd48c003a50 0x7fd4a015db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a7c2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4a01e96f0 0x7fd4a00ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a7c2a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a015bd70 0x7fd4a00cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a7c2a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a00d01e0 con 0x7fd4a01e96f0 2024-05-11T01:01:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a7c2a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a00cff00 con 0x7fd4a015bd70 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a7c2a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a00b4ee0 con 0x7fd48c003a50 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a51c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4a01e96f0 0x7fd4a00ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a51c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4a01e96f0 0x7fd4a00ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40780/0 (socket says 172.21.15.52:40780) 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.078+0000 7fd4a51c5640 1 -- 172.21.15.52:0/307439340 learned_addr learned my addr 172.21.15.52:0/307439340 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a51c5640 1 -- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd48c003a50 msgr2=0x7fd4a015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a51c5640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd48c003a50 0x7fd4a015db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a51c5640 1 -- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a015bd70 msgr2=0x7fd4a00cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a61c7640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a015bd70 0x7fd4a00cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a51c5640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a015bd70 0x7fd4a00cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a51c5640 1 -- 172.21.15.52:0/307439340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a00cea60 con 0x7fd4a01e96f0 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a61c7640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a015bd70 0x7fd4a00cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a51c5640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4a01e96f0 0x7fd4a00ce520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd49000e780 tx=0x7fd49000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4967fc640 1 -- 172.21.15.52:0/307439340 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4900050d0 con 0x7fd4a01e96f0 2024-05-11T01:01:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4967fc640 1 -- 172.21.15.52:0/307439340 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd490018070 con 0x7fd4a01e96f0 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a00cd480 con 0x7fd4a01e96f0 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4967fc640 1 -- 172.21.15.52:0/307439340 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490005490 con 0x7fd4a01e96f0 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.079+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a00cda40 con 0x7fd4a01e96f0 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.080+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a00695d0 con 0x7fd4a01e96f0 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.081+0000 7fd4967fc640 1 -- 172.21.15.52:0/307439340 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd49000c6c0 con 0x7fd4a01e96f0 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.081+0000 7fd4967fc640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd484076300 0x7fd4840787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.081+0000 7fd4967fc640 1 -- 172.21.15.52:0/307439340 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fd490097320 con 0x7fd4a01e96f0 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.081+0000 7fd4a59c6640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd484076300 0x7fd4840787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.082+0000 7fd4a59c6640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd484076300 0x7fd4840787c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd49c033000 tx=0x7fd49c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.086+0000 7fd4967fc640 1 -- 172.21.15.52:0/307439340 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd490060c80 con 0x7fd4a01e96f0 2024-05-11T01:01:46.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:46.319+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd4a00d0b70 con 0x7fd4a01e96f0 2024-05-11T01:01:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.294+0000 7fd4967fc640 1 -- 172.21.15.52:0/307439340 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fd490060580 con 0x7fd4a01e96f0 2024-05-11T01:01:47.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd484076300 msgr2=0x7fd4840787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd484076300 0x7fd4840787c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd49c033000 tx=0x7fd49c02e000 comp rx=0 tx=0).stop 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4a01e96f0 msgr2=0x7fd4a00ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4a01e96f0 0x7fd4a00ce520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd49000e780 tx=0x7fd49000ec40 comp rx=0 tx=0).stop 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 shutdown_connections 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd484076300 0x7fd4840787c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a015bd70 0x7fd4a00cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4a01e96f0 0x7fd4a00ce520 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 --2- 172.21.15.52:0/307439340 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd48c003a50 0x7fd4a015db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.297+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 >> 172.21.15.52:0/307439340 conn(0x7fd4a008ebb0 msgr2=0x7fd4a00b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.298+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 shutdown_connections 2024-05-11T01:01:47.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.298+0000 7fd4a7c2a640 1 -- 172.21.15.52:0/307439340 wait complete. 2024-05-11T01:01:47.311 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T01:01:47.311 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T01:01:47.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.425+0000 7fb7b6408640 1 Processor -- start 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.425+0000 7fb7b6408640 1 -- start start 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.425+0000 7fb7b6408640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb7b015f2f0 0x7fb7b00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.425+0000 7fb7b6408640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb7b00d4e40 0x7fb7b00d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7b6408640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b00d3a90 con 0x7fb7b00d2d60 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7b6408640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b00d3d70 con 0x7fb7b015f2f0 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7b6408640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b00d4050 con 0x7fb7b00d4e40 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7affff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb7b015f2f0 0x7fb7b00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7affff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb7b015f2f0 0x7fb7b00cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40794/0 (socket says 172.21.15.52:40794) 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7affff640 1 -- 172.21.15.52:0/2755262265 learned_addr learned my addr 172.21.15.52:0/2755262265 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7b49a5640 1 --1- 172.21.15.52:0/2755262265 >> v1:172.21.15.52:6789/0 conn(0x7fb7b00d2d60 0x7fb7b00d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33846/0 (socket says 172.21.15.52:33846) 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7af7fe640 1 --2- 172.21.15.52:0/2755262265 >> v2:172.21.15.119:3300/0 conn(0x7fb7b00d4e40 0x7fb7b00d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:47.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7affff640 1 -- 172.21.15.52:0/2755262265 >> v2:172.21.15.119:3300/0 conn(0x7fb7b00d4e40 msgr2=0x7fb7b00d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7affff640 1 --2- 172.21.15.52:0/2755262265 >> v2:172.21.15.119:3300/0 conn(0x7fb7b00d4e40 0x7fb7b00d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7affff640 1 -- 172.21.15.52:0/2755262265 >> v1:172.21.15.52:6789/0 conn(0x7fb7b00d2d60 legacy=0x7fb7b00d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7affff640 1 -- 172.21.15.52:0/2755262265 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b00b4e60 con 0x7fb7b015f2f0 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7af7fe640 1 --2- 172.21.15.52:0/2755262265 >> v2:172.21.15.119:3300/0 conn(0x7fb7b00d4e40 0x7fb7b00d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.426+0000 7fb7aeffd640 1 -- 172.21.15.52:0/2755262265 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2033907947 0 0) 0x7fb7b00d3a90 con 0x7fb7b00d2d60 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7affff640 1 --2- 172.21.15.52:0/2755262265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 0x7fb7b00cfca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00b4910 tx=0x7fb7a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23552468c83ea73f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7aeffd640 1 -- 172.21.15.52:0/2755262265 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0037070 con 0x7fb7b015f2f0 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7aeffd640 1 -- 172.21.15.52:0/2755262265 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a0032440 con 0x7fb7b015f2f0 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7aeffd640 1 -- 172.21.15.52:0/2755262265 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0032810 con 0x7fb7b015f2f0 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7b6408640 1 -- 172.21.15.52:0/2755262265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 msgr2=0x7fb7b00cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7b6408640 1 --2- 172.21.15.52:0/2755262265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 0x7fb7b00cfca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00b4910 tx=0x7fb7a002a270 comp rx=0 tx=0).stop 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7b6408640 1 -- 172.21.15.52:0/2755262265 shutdown_connections 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7b6408640 1 --2- 172.21.15.52:0/2755262265 >> v2:172.21.15.119:3300/0 conn(0x7fb7b00d4e40 0x7fb7b00d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:47.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7b6408640 1 --2- 172.21.15.52:0/2755262265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 0x7fb7b00cfca0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:47.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.427+0000 7fb7b6408640 1 -- 172.21.15.52:0/2755262265 >> 172.21.15.52:0/2755262265 conn(0x7fb7b008ebb0 msgr2=0x7fb7b009e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:47.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.428+0000 7fb7b6408640 1 -- 172.21.15.52:0/2755262265 shutdown_connections 2024-05-11T01:01:47.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.428+0000 7fb7b6408640 1 -- 172.21.15.52:0/2755262265 wait complete. 2024-05-11T01:01:47.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.429+0000 7fb7b6408640 1 Processor -- start 2024-05-11T01:01:47.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.429+0000 7fb7b6408640 1 -- start start 2024-05-11T01:01:47.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b6408640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b00d2d60 0x7fb7b015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:47.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b6408640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7b00d4e40 0x7fb7b00d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:47.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b6408640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 0x7fb7b00d1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:47.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b6408640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b00d01e0 con 0x7fb7b015f2f0 2024-05-11T01:01:47.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b6408640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b00d57d0 con 0x7fb7b00d2d60 2024-05-11T01:01:47.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b6408640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b00d0360 con 0x7fb7b00d4e40 2024-05-11T01:01:47.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b49a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 0x7fb7b00d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b49a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 0x7fb7b00d1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40802/0 (socket says 172.21.15.52:40802) 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b49a5640 1 -- 172.21.15.52:0/1929873797 learned_addr learned my addr 172.21.15.52:0/1929873797 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7affff640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b00d2d60 0x7fb7b015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7af7fe640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7b00d4e40 0x7fb7b00d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b49a5640 1 -- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7b00d4e40 msgr2=0x7fb7b00d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b49a5640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7b00d4e40 0x7fb7b00d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b49a5640 1 -- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b00d2d60 msgr2=0x7fb7b015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b49a5640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b00d2d60 0x7fb7b015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b49a5640 1 -- 172.21.15.52:0/1929873797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b015ce70 con 0x7fb7b015f2f0 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7affff640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b00d2d60 0x7fb7b015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b49a5640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 0x7fb7b00d1710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800c6b0 tx=0x7fb7a800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7acff9640 1 -- 172.21.15.52:0/1929873797 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8018070 con 0x7fb7b015f2f0 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7acff9640 1 -- 172.21.15.52:0/1929873797 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a8004da0 con 0x7fb7b015f2f0 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7acff9640 1 -- 172.21.15.52:0/1929873797 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800a880 con 0x7fb7b015f2f0 2024-05-11T01:01:47.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.430+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b00a1ac0 con 0x7fb7b015f2f0 2024-05-11T01:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.431+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b00a2080 con 0x7fb7b015f2f0 2024-05-11T01:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.431+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b00695d0 con 0x7fb7b015f2f0 2024-05-11T01:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.432+0000 7fb7acff9640 1 -- 172.21.15.52:0/1929873797 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb7a801b070 con 0x7fb7b015f2f0 2024-05-11T01:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.433+0000 7fb7acff9640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb79c0762d0 0x7fb79c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.433+0000 7fb7acff9640 1 -- 172.21.15.52:0/1929873797 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fb7a8096c90 con 0x7fb7b015f2f0 2024-05-11T01:01:47.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.433+0000 7fb7affff640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb79c0762d0 0x7fb79c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:47.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.434+0000 7fb7affff640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb79c0762d0 0x7fb79c078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0034000 tx=0x7fb7a002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:47.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.437+0000 7fb7acff9640 1 -- 172.21.15.52:0/1929873797 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb7a8060550 con 0x7fb7b015f2f0 2024-05-11T01:01:47.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:47.668+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb7b006edb0 con 0x7fb7b015f2f0 2024-05-11T01:01:48.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.299+0000 7fb7acff9640 1 -- 172.21.15.52:0/1929873797 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fb7a805fe50 con 0x7fb7b015f2f0 2024-05-11T01:01:48.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb79c0762d0 msgr2=0x7fb79c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb79c0762d0 0x7fb79c078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0034000 tx=0x7fb7a002e000 comp rx=0 tx=0).stop 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 msgr2=0x7fb7b00d1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 0x7fb7b00d1710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800c6b0 tx=0x7fb7a800cb70 comp rx=0 tx=0).stop 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 shutdown_connections 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb79c0762d0 0x7fb79c078790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b015f2f0 0x7fb7b00d1710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb7b00d4e40 0x7fb7b00d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 --2- 172.21.15.52:0/1929873797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7b00d2d60 0x7fb7b015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 >> 172.21.15.52:0/1929873797 conn(0x7fb7b008ebb0 msgr2=0x7fb7b00b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 shutdown_connections 2024-05-11T01:01:48.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.302+0000 7fb7b6408640 1 -- 172.21.15.52:0/1929873797 wait complete. 2024-05-11T01:01:48.316 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T01:01:48.316 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:48.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.425+0000 7f9368447640 1 Processor -- start 2024-05-11T01:01:48.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.425+0000 7f9368447640 1 -- start start 2024-05-11T01:01:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.426+0000 7f9368447640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93600b5dd0 0x7f93600d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.426+0000 7f9368447640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9360163250 con 0x7f9360160480 2024-05-11T01:01:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.426+0000 7f9368447640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9360163530 con 0x7f936015f430 2024-05-11T01:01:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.426+0000 7f9368447640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9360163810 con 0x7f93600b5dd0 2024-05-11T01:01:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.427+0000 7f93659e2640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f9360160480 0x7f93600d2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33850/0 (socket says 172.21.15.52:33850) 2024-05-11T01:01:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.427+0000 7f93659e2640 1 -- 172.21.15.52:0/3544462728 learned_addr learned my addr 172.21.15.52:0/3544462728 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.427+0000 7f93669e4640 1 --2- 172.21.15.52:0/3544462728 >> v2:172.21.15.119:3300/0 conn(0x7f93600b5dd0 0x7f93600d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.427+0000 7f93651e1640 1 -- 172.21.15.52:0/3544462728 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1811680158 0 0) 0x7f9360163250 con 0x7f9360160480 2024-05-11T01:01:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.427+0000 7f93651e1640 1 -- 172.21.15.52:0/3544462728 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9348003520 con 0x7f9360160480 2024-05-11T01:01:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.427+0000 7f93651e1640 1 -- 172.21.15.52:0/3544462728 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3242796956 0 0) 0x7f9348003520 con 0x7f9360160480 2024-05-11T01:01:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.427+0000 7f93669e4640 1 -- 172.21.15.52:0/3544462728 >> v1:172.21.15.119:6789/0 conn(0x7f936015f430 legacy=0x7f936015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.427+0000 7f93669e4640 1 -- 172.21.15.52:0/3544462728 >> v1:172.21.15.52:6789/0 conn(0x7f9360160480 legacy=0x7f93600d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.427+0000 7f93669e4640 1 -- 172.21.15.52:0/3544462728 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9360163af0 con 0x7f93600b5dd0 2024-05-11T01:01:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.428+0000 7f93669e4640 1 --2- 172.21.15.52:0/3544462728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600b5dd0 0x7f93600d3490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f93580097a0 tx=0x7f935802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41e50db4423a5c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.428+0000 7f93651e1640 1 -- 172.21.15.52:0/3544462728 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358037070 con 0x7f93600b5dd0 2024-05-11T01:01:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.428+0000 7f93651e1640 1 -- 172.21.15.52:0/3544462728 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935802abc0 con 0x7f93600b5dd0 2024-05-11T01:01:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.428+0000 7f93651e1640 1 -- 172.21.15.52:0/3544462728 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93540016f0 con 0x7f93600b5dd0 2024-05-11T01:01:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.429+0000 7f9368447640 1 -- 172.21.15.52:0/3544462728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600b5dd0 msgr2=0x7f93600d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.429+0000 7f9368447640 1 --2- 172.21.15.52:0/3544462728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600b5dd0 0x7f93600d3490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f93580097a0 tx=0x7f935802a130 comp rx=0 tx=0).stop 2024-05-11T01:01:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.429+0000 7f9368447640 1 -- 172.21.15.52:0/3544462728 shutdown_connections 2024-05-11T01:01:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.429+0000 7f9368447640 1 --2- 172.21.15.52:0/3544462728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600b5dd0 0x7f93600d3490 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.429+0000 7f9368447640 1 -- 172.21.15.52:0/3544462728 >> 172.21.15.52:0/3544462728 conn(0x7f936008eb70 msgr2=0x7f936009e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.429+0000 7f9368447640 1 -- 172.21.15.52:0/3544462728 shutdown_connections 2024-05-11T01:01:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.429+0000 7f9368447640 1 -- 172.21.15.52:0/3544462728 wait complete. 2024-05-11T01:01:48.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.430+0000 7f9368447640 1 Processor -- start 2024-05-11T01:01:48.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.430+0000 7f9368447640 1 -- start start 2024-05-11T01:01:48.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f9368447640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93600b5dd0 0x7f936015a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f9368447640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936015f430 0x7f93601f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f9368447640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9360160480 0x7f93600ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f9368447640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9360163030 con 0x7f93600b5dd0 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f9368447640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9360162eb0 con 0x7f936015f430 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f9368447640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f93601608f0 con 0x7f9360160480 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93661e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93600b5dd0 0x7f936015a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93661e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93600b5dd0 0x7f936015a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40816/0 (socket says 172.21.15.52:40816) 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93661e3640 1 -- 172.21.15.52:0/4246842867 learned_addr learned my addr 172.21.15.52:0/4246842867 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93659e2640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936015f430 0x7f93601f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93669e4640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9360160480 0x7f93600ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93661e3640 1 -- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9360160480 msgr2=0x7f93600ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93661e3640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9360160480 0x7f93600ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93661e3640 1 -- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936015f430 msgr2=0x7f93601f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93661e3640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936015f430 0x7f93601f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93661e3640 1 -- 172.21.15.52:0/4246842867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93601f93c0 con 0x7f93600b5dd0 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.431+0000 7f93669e4640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9360160480 0x7f93600ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.432+0000 7f93661e3640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93600b5dd0 0x7f936015a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f935000c8a0 tx=0x7f935000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.432+0000 7f935effd640 1 -- 172.21.15.52:0/4246842867 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350003b70 con 0x7f93600b5dd0 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.432+0000 7f935effd640 1 -- 172.21.15.52:0/4246842867 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9350003d10 con 0x7f93600b5dd0 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.432+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93600ce5e0 con 0x7f93600b5dd0 2024-05-11T01:01:48.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.432+0000 7f935effd640 1 -- 172.21.15.52:0/4246842867 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935000aa70 con 0x7f93600b5dd0 2024-05-11T01:01:48.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.432+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93601f8a90 con 0x7f93600b5dd0 2024-05-11T01:01:48.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.433+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9360069780 con 0x7f93600b5dd0 2024-05-11T01:01:48.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.434+0000 7f935effd640 1 -- 172.21.15.52:0/4246842867 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f935000ac10 con 0x7f93600b5dd0 2024-05-11T01:01:48.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.434+0000 7f935effd640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f934c0762d0 0x7f934c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.434+0000 7f935effd640 1 -- 172.21.15.52:0/4246842867 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f9350097590 con 0x7f93600b5dd0 2024-05-11T01:01:48.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.434+0000 7f93659e2640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f934c0762d0 0x7f934c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.435+0000 7f93659e2640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f934c0762d0 0x7f934c078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f93601f5fe0 tx=0x7f9354009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:48.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.439+0000 7f935effd640 1 -- 172.21.15.52:0/4246842867 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9350060f00 con 0x7f93600b5dd0 2024-05-11T01:01:48.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.670+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93600b4ba0 con 0x7f93600b5dd0 2024-05-11T01:01:48.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.670+0000 7f935effd640 1 -- 172.21.15.52:0/4246842867 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7f9350060800 con 0x7f93600b5dd0 2024-05-11T01:01:48.671 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:48.671 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3122737368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3122737368},{"type":"v1","addr":"172.21.15.119:6836","nonce":3122737368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/211602681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":211602681},{"type":"v1","addr":"172.21.15.119:6837","nonce":211602681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2079689543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2079689543},{"type":"v1","addr":"172.21.15.52:6835","nonce":2079689543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:44.660501+0000","modified":"2024-05-11T01:01:47.669355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4854":{"gid":4854,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1325827549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1325827549},{"type":"v1","addr":"172.21.15.52:6834","nonce":1325827549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4854,"qdb_cluster":[4854]},"id":2}]} 2024-05-11T01:01:48.671 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-05-11T01:01:48.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f934c0762d0 msgr2=0x7f934c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f934c0762d0 0x7f934c078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f93601f5fe0 tx=0x7f9354009040 comp rx=0 tx=0).stop 2024-05-11T01:01:48.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93600b5dd0 msgr2=0x7f936015a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93600b5dd0 0x7f936015a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f935000c8a0 tx=0x7f935000cd60 comp rx=0 tx=0).stop 2024-05-11T01:01:48.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 shutdown_connections 2024-05-11T01:01:48.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f934c0762d0 0x7f934c078790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9360160480 0x7f93600ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936015f430 0x7f93601f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 --2- 172.21.15.52:0/4246842867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93600b5dd0 0x7f936015a7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.672+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 >> 172.21.15.52:0/4246842867 conn(0x7f936008eb70 msgr2=0x7f93600b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:48.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.673+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 shutdown_connections 2024-05-11T01:01:48.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.673+0000 7f9368447640 1 -- 172.21.15.52:0/4246842867 wait complete. 2024-05-11T01:01:48.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:01:48.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.788+0000 7f6492815640 1 Processor -- start 2024-05-11T01:01:48.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.788+0000 7f6492815640 1 -- start start 2024-05-11T01:01:48.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.789+0000 7f6492815640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f648c072ce0 0x7f648c06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.789+0000 7f6492815640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 0x7f648c165b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.789+0000 7f6492815640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f648c0aea90 con 0x7f648c0afb30 2024-05-11T01:01:48.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.789+0000 7f6492815640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f648c0b59e0 con 0x7f648c072ce0 2024-05-11T01:01:48.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.789+0000 7f6492815640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f648c0ae860 con 0x7f648c0737d0 2024-05-11T01:01:48.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.789+0000 7f648b7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f648c0afb30 0x7f648c0730c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33864/0 (socket says 172.21.15.52:33864) 2024-05-11T01:01:48.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.789+0000 7f648b7fe640 1 -- 172.21.15.52:0/2096909649 learned_addr learned my addr 172.21.15.52:0/2096909649 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:48.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.789+0000 7f648bfff640 1 --2- 172.21.15.52:0/2096909649 >> v2:172.21.15.119:3300/0 conn(0x7f648c072ce0 0x7f648c06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.789+0000 7f6490db2640 1 --2- 172.21.15.52:0/2096909649 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 0x7f648c165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.790+0000 7f648affd640 1 -- 172.21.15.52:0/2096909649 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4010323849 0 0) 0x7f648c0aea90 con 0x7f648c0afb30 2024-05-11T01:01:48.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.790+0000 7f648affd640 1 -- 172.21.15.52:0/2096909649 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6474003520 con 0x7f648c0afb30 2024-05-11T01:01:48.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.790+0000 7f648affd640 1 -- 172.21.15.52:0/2096909649 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3792480599 0 0) 0x7f6474003520 con 0x7f648c0afb30 2024-05-11T01:01:48.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.790+0000 7f648bfff640 1 -- 172.21.15.52:0/2096909649 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 msgr2=0x7f648c165b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:48.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.790+0000 7f648bfff640 1 --2- 172.21.15.52:0/2096909649 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 0x7f648c165b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.790+0000 7f648bfff640 1 -- 172.21.15.52:0/2096909649 >> v1:172.21.15.52:6789/0 conn(0x7f648c0afb30 legacy=0x7f648c0730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.790+0000 7f648bfff640 1 -- 172.21.15.52:0/2096909649 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c0aed70 con 0x7f648c072ce0 2024-05-11T01:01:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f648bfff640 1 --2- 172.21.15.52:0/2096909649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648c072ce0 0x7f648c06e050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f647c0097a0 tx=0x7f647c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c777fa5c0dcecfb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f648affd640 1 -- 172.21.15.52:0/2096909649 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c037070 con 0x7f648c072ce0 2024-05-11T01:01:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f648affd640 1 -- 172.21.15.52:0/2096909649 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f647c02aba0 con 0x7f648c072ce0 2024-05-11T01:01:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f648affd640 1 -- 172.21.15.52:0/2096909649 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64800016f0 con 0x7f648c072ce0 2024-05-11T01:01:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f6492815640 1 -- 172.21.15.52:0/2096909649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648c072ce0 msgr2=0x7f648c06e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f6492815640 1 --2- 172.21.15.52:0/2096909649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648c072ce0 0x7f648c06e050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f647c0097a0 tx=0x7f647c02a110 comp rx=0 tx=0).stop 2024-05-11T01:01:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f6492815640 1 -- 172.21.15.52:0/2096909649 shutdown_connections 2024-05-11T01:01:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f6492815640 1 --2- 172.21.15.52:0/2096909649 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 0x7f648c165b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f6492815640 1 --2- 172.21.15.52:0/2096909649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648c072ce0 0x7f648c06e050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f6492815640 1 -- 172.21.15.52:0/2096909649 >> 172.21.15.52:0/2096909649 conn(0x7f648c0b2a00 msgr2=0x7f648c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f6492815640 1 -- 172.21.15.52:0/2096909649 shutdown_connections 2024-05-11T01:01:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.791+0000 7f6492815640 1 -- 172.21.15.52:0/2096909649 wait complete. 2024-05-11T01:01:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.793+0000 7f6492815640 1 Processor -- start 2024-05-11T01:01:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.793+0000 7f6492815640 1 -- start start 2024-05-11T01:01:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6492815640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648c072ce0 0x7f648c146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6492815640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 0x7f648c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6492815640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f648c0afb30 0x7f648c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6492815640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f648c0af610 con 0x7f648c0afb30 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6492815640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f648c0af330 con 0x7f648c072ce0 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6492815640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f648c06e590 con 0x7f648c0737d0 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6490db2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f648c0afb30 0x7f648c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6490db2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f648c0afb30 0x7f648c0b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40818/0 (socket says 172.21.15.52:40818) 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6490db2640 1 -- 172.21.15.52:0/838518947 learned_addr learned my addr 172.21.15.52:0/838518947 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f648bfff640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648c072ce0 0x7f648c146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f648b7fe640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 0x7f648c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6490db2640 1 -- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 msgr2=0x7f648c1533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6490db2640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 0x7f648c1533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6490db2640 1 -- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648c072ce0 msgr2=0x7f648c146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6490db2640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648c072ce0 0x7f648c146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6490db2640 1 -- 172.21.15.52:0/838518947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c147020 con 0x7f648c0afb30 2024-05-11T01:01:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f648b7fe640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 0x7f648c1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6490db2640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f648c0afb30 0x7f648c0b3d50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f648400efc0 tx=0x7f648400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6488ff9640 1 -- 172.21.15.52:0/838518947 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484018020 con 0x7f648c0afb30 2024-05-11T01:01:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6488ff9640 1 -- 172.21.15.52:0/838518947 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6484014480 con 0x7f648c0afb30 2024-05-11T01:01:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c156d20 con 0x7f648c0afb30 2024-05-11T01:01:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.794+0000 7f6488ff9640 1 -- 172.21.15.52:0/838518947 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484014870 con 0x7f648c0afb30 2024-05-11T01:01:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.795+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c1572e0 con 0x7f648c0afb30 2024-05-11T01:01:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.796+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648c069780 con 0x7f648c0afb30 2024-05-11T01:01:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.796+0000 7f6488ff9640 1 -- 172.21.15.52:0/838518947 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6484005d10 con 0x7f648c0afb30 2024-05-11T01:01:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.797+0000 7f6488ff9640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6478076300 0x7f64780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.797+0000 7f6488ff9640 1 -- 172.21.15.52:0/838518947 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f64840982d0 con 0x7f648c0afb30 2024-05-11T01:01:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.797+0000 7f648bfff640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6478076300 0x7f64780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.798+0000 7f648bfff640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6478076300 0x7f64780787c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f647c034000 tx=0x7f647c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:48.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.801+0000 7f6488ff9640 1 -- 172.21.15.52:0/838518947 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6484022030 con 0x7f648c0afb30 2024-05-11T01:01:48.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.994+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f648c06f7d0 con 0x7f648c0afb30 2024-05-11T01:01:48.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.995+0000 7f6488ff9640 1 -- 172.21.15.52:0/838518947 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16669 (secure 0 0 0) 0x7f6484010040 con 0x7f648c0afb30 2024-05-11T01:01:48.995 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:48.995 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":31,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:01:46.280852+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:01:42.710844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:43.612754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:01:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6478076300 msgr2=0x7f64780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6478076300 0x7f64780787c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f647c034000 tx=0x7f647c02e000 comp rx=0 tx=0).stop 2024-05-11T01:01:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f648c0afb30 msgr2=0x7f648c0b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f648c0afb30 0x7f648c0b3d50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f648400efc0 tx=0x7f648400c490 comp rx=0 tx=0).stop 2024-05-11T01:01:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 shutdown_connections 2024-05-11T01:01:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f6478076300 0x7f64780787c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f648c0afb30 0x7f648c0b3d50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f648c0737d0 0x7f648c1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 --2- 172.21.15.52:0/838518947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648c072ce0 0x7f648c146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:48.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 >> 172.21.15.52:0/838518947 conn(0x7f648c0b2a00 msgr2=0x7f648c0730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:48.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 shutdown_connections 2024-05-11T01:01:48.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:48.997+0000 7f6492815640 1 -- 172.21.15.52:0/838518947 wait complete. 2024-05-11T01:01:49.008 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T01:01:49.008 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:01:49.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.112+0000 7f638d3da640 1 Processor -- start 2024-05-11T01:01:49.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.112+0000 7f638d3da640 1 -- start start 2024-05-11T01:01:49.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f638d3da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f638815f430 0x7f638815fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f638d3da640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f63881602b0 0x7f63881606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f638d3da640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f63880d1b50 0x7f63880d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f638d3da640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f6388163250 con 0x7f63881602b0 2024-05-11T01:01:49.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f638d3da640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6388163530 con 0x7f63880d1b50 2024-05-11T01:01:49.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f638d3da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6388163810 con 0x7f638815f430 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f63867fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f63881602b0 0x7f63881606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f63867fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f63881602b0 0x7f63881606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40826/0 (socket says 172.21.15.52:40826) 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f63867fc640 1 -- 172.21.15.52:0/3990669893 learned_addr learned my addr 172.21.15.52:0/3990669893 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f6386ffd640 1 --2- 172.21.15.52:0/3990669893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f638815f430 0x7f638815fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f63877fe640 1 --2- 172.21.15.52:0/3990669893 >> v2:172.21.15.119:3300/0 conn(0x7f63880d1b50 0x7f63880d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f63867fc640 1 -- 172.21.15.52:0/3990669893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f638815f430 msgr2=0x7f638815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f63867fc640 1 --2- 172.21.15.52:0/3990669893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f638815f430 0x7f638815fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f63867fc640 1 -- 172.21.15.52:0/3990669893 >> v2:172.21.15.119:3300/0 conn(0x7f63880d1b50 msgr2=0x7f63880d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f63867fc640 1 --2- 172.21.15.52:0/3990669893 >> v2:172.21.15.119:3300/0 conn(0x7f63880d1b50 0x7f63880d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f63867fc640 1 -- 172.21.15.52:0/3990669893 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6388163af0 con 0x7f63881602b0 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.113+0000 7f6386ffd640 1 --2- 172.21.15.52:0/3990669893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f638815f430 0x7f638815fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.114+0000 7f63867fc640 1 --2- 172.21.15.52:0/3990669893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63881602b0 0x7f63881606f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f63880cf710 tx=0x7f637002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7a47a50c037cd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.114+0000 7f6385ffb640 1 -- 172.21.15.52:0/3990669893 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370037070 con 0x7f63881602b0 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.114+0000 7f6385ffb640 1 -- 172.21.15.52:0/3990669893 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6370032440 con 0x7f63881602b0 2024-05-11T01:01:49.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.114+0000 7f6385ffb640 1 -- 172.21.15.52:0/3990669893 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370032810 con 0x7f63881602b0 2024-05-11T01:01:49.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.115+0000 7f638d3da640 1 -- 172.21.15.52:0/3990669893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63881602b0 msgr2=0x7f63881606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.115+0000 7f638d3da640 1 --2- 172.21.15.52:0/3990669893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63881602b0 0x7f63881606f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f63880cf710 tx=0x7f637002a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.115+0000 7f638d3da640 1 -- 172.21.15.52:0/3990669893 shutdown_connections 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.115+0000 7f638d3da640 1 --2- 172.21.15.52:0/3990669893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63881602b0 0x7f63881606f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.115+0000 7f638d3da640 1 --2- 172.21.15.52:0/3990669893 >> v2:172.21.15.119:3300/0 conn(0x7f63880d1b50 0x7f63880d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.115+0000 7f638d3da640 1 --2- 172.21.15.52:0/3990669893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f638815f430 0x7f638815fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.115+0000 7f638d3da640 1 -- 172.21.15.52:0/3990669893 >> 172.21.15.52:0/3990669893 conn(0x7f638808eb70 msgr2=0x7f638809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.115+0000 7f638d3da640 1 -- 172.21.15.52:0/3990669893 shutdown_connections 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.115+0000 7f638d3da640 1 -- 172.21.15.52:0/3990669893 wait complete. 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.116+0000 7f638d3da640 1 Processor -- start 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.116+0000 7f638d3da640 1 -- start start 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f638d3da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63880d1b50 0x7f638815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f638d3da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638815f430 0x7f63880c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f638d3da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63881602b0 0x7f63880ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f638d3da640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6388163030 con 0x7f638815f430 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f638d3da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6388162eb0 con 0x7f63881602b0 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f638d3da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f63880d2540 con 0x7f63880d1b50 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63867fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638815f430 0x7f63880c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63867fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638815f430 0x7f63880c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40842/0 (socket says 172.21.15.52:40842) 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63867fc640 1 -- 172.21.15.52:0/540870582 learned_addr learned my addr 172.21.15.52:0/540870582 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f6386ffd640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63880d1b50 0x7f638815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63877fe640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63881602b0 0x7f63880ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63867fc640 1 -- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63880d1b50 msgr2=0x7f638815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63867fc640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63880d1b50 0x7f638815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63867fc640 1 -- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63881602b0 msgr2=0x7f63880ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63867fc640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63881602b0 0x7f63880ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63867fc640 1 -- 172.21.15.52:0/540870582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63880b7d30 con 0x7f638815f430 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.117+0000 7f63877fe640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63881602b0 0x7f63880ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.118+0000 7f63867fc640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638815f430 0x7f63880c8c70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6370000c00 tx=0x7f63700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:49.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.118+0000 7f637f7fe640 1 -- 172.21.15.52:0/540870582 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370041020 con 0x7f638815f430 2024-05-11T01:01:49.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.118+0000 7f637f7fe640 1 -- 172.21.15.52:0/540870582 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63700306c0 con 0x7f638815f430 2024-05-11T01:01:49.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.118+0000 7f637f7fe640 1 -- 172.21.15.52:0/540870582 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370037040 con 0x7f638815f430 2024-05-11T01:01:49.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.118+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63880c9c60 con 0x7f638815f430 2024-05-11T01:01:49.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.118+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63880c9980 con 0x7f638815f430 2024-05-11T01:01:49.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.119+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f638809e9c0 con 0x7f638815f430 2024-05-11T01:01:49.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.119+0000 7f637f7fe640 1 -- 172.21.15.52:0/540870582 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6370044020 con 0x7f638815f430 2024-05-11T01:01:49.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.120+0000 7f637f7fe640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f63680762d0 0x7f6368078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.120+0000 7f637f7fe640 1 -- 172.21.15.52:0/540870582 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f63700b7da0 con 0x7f638815f430 2024-05-11T01:01:49.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.120+0000 7f6386ffd640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f63680762d0 0x7f6368078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.121+0000 7f6386ffd640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f63680762d0 0x7f6368078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63880d2070 tx=0x7f6380003a10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:49.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.124+0000 7f637f7fe640 1 -- 172.21.15.52:0/540870582 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f638809e9c0 con 0x7f638815f430 2024-05-11T01:01:49.252 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-11T01:01:49.252+0000 7f77fe539640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T01:01:49.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.312+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6388069780 con 0x7f63680762d0 2024-05-11T01:01:49.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.318+0000 7f637f7fe640 1 -- 172.21.15.52:0/540870582 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7f6388069780 con 0x7f63680762d0 2024-05-11T01:01:49.321 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:49.322 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:01:49.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f63680762d0 msgr2=0x7f6368078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f63680762d0 0x7f6368078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63880d2070 tx=0x7f6380003a10 comp rx=0 tx=0).stop 2024-05-11T01:01:49.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638815f430 msgr2=0x7f63880c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638815f430 0x7f63880c8c70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6370000c00 tx=0x7f63700046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 shutdown_connections 2024-05-11T01:01:49.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f63680762d0 0x7f6368078790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63881602b0 0x7f63880ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f638815f430 0x7f63880c8c70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 --2- 172.21.15.52:0/540870582 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63880d1b50 0x7f638815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 >> 172.21.15.52:0/540870582 conn(0x7f638808eb70 msgr2=0x7f63880d3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:49.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 shutdown_connections 2024-05-11T01:01:49.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.321+0000 7f638d3da640 1 -- 172.21.15.52:0/540870582 wait complete. 2024-05-11T01:01:49.332 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-05-11T01:01:47.520430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236584,"kb_used_data":3776,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738136,"statfs":{"total":773094113280,"available":772851851264,"internally_reserved":0,"allocated":3866624,"data_stored":2177030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.252285"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308416+0000","last_change":"2024-05-11T01:01:45.292273+0000","last_active":"2024-05-11T01:01:46.308416+0000","last_peered":"2024-05-11T01:01:46.308416+0000","last_clean":"2024-05-11T01:01:46.308416+0000","last_became_active":"2024-05-11T01:01:45.292019+0000","last_became_peered":"2024-05-11T01:01:45.292019+0000","last_unstale":"2024-05-11T01:01:46.308416+0000","last_undegraded":"2024-05-11T01:01:46.308416+0000","last_fullsized":"2024-05-11T01:01:46.308416+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:24.197518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.295181+0000","last_change":"2024-05-11T01:01:45.289886+0000","last_active":"2024-05-11T01:01:45.295181+0000","last_peered":"2024-05-11T01:01:45.295181+0000","last_clean":"2024-05-11T01:01:45.295181+0000","last_became_active":"2024-05-11T01:01:45.289413+0000","last_became_peered":"2024-05-11T01:01:45.289413+0000","last_unstale":"2024-05-11T01:01:45.295181+0000","last_undegraded":"2024-05-11T01:01:45.295181+0000","last_fullsized":"2024-05-11T01:01:45.295181+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:11:53.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289939+0000","last_change":"2024-05-11T01:01:45.286186+0000","last_active":"2024-05-11T01:01:45.289939+0000","last_peered":"2024-05-11T01:01:45.289939+0000","last_clean":"2024-05-11T01:01:45.289939+0000","last_became_active":"2024-05-11T01:01:45.285897+0000","last_became_peered":"2024-05-11T01:01:45.285897+0000","last_unstale":"2024-05-11T01:01:45.289939+0000","last_undegraded":"2024-05-11T01:01:45.289939+0000","last_fullsized":"2024-05-11T01:01:45.289939+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:23:44.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.296135+0000","last_change":"2024-05-11T01:01:45.292553+0000","last_active":"2024-05-11T01:01:45.296135+0000","last_peered":"2024-05-11T01:01:45.296135+0000","last_clean":"2024-05-11T01:01:45.296135+0000","last_became_active":"2024-05-11T01:01:45.292224+0000","last_became_peered":"2024-05-11T01:01:45.292224+0000","last_unstale":"2024-05-11T01:01:45.296135+0000","last_undegraded":"2024-05-11T01:01:45.296135+0000","last_fullsized":"2024-05-11T01:01:45.296135+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:24.197518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301408+0000","last_change":"2024-05-11T01:01:45.298505+0000","last_active":"2024-05-11T01:01:45.301408+0000","last_peered":"2024-05-11T01:01:45.301408+0000","last_clean":"2024-05-11T01:01:45.301408+0000","last_became_active":"2024-05-11T01:01:45.298012+0000","last_became_peered":"2024-05-11T01:01:45.298012+0000","last_unstale":"2024-05-11T01:01:45.301408+0000","last_undegraded":"2024-05-11T01:01:45.301408+0000","last_fullsized":"2024-05-11T01:01:45.301408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:56.599542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305466+0000","last_change":"2024-05-11T01:01:45.299253+0000","last_active":"2024-05-11T01:01:45.305466+0000","last_peered":"2024-05-11T01:01:45.305466+0000","last_clean":"2024-05-11T01:01:45.305466+0000","last_became_active":"2024-05-11T01:01:45.298975+0000","last_became_peered":"2024-05-11T01:01:45.298975+0000","last_unstale":"2024-05-11T01:01:45.305466+0000","last_undegraded":"2024-05-11T01:01:45.305466+0000","last_fullsized":"2024-05-11T01:01:45.305466+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:17:32.179452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.298813+0000","last_change":"2024-05-11T01:01:45.295748+0000","last_active":"2024-05-11T01:01:45.298813+0000","last_peered":"2024-05-11T01:01:45.298813+0000","last_clean":"2024-05-11T01:01:45.298813+0000","last_became_active":"2024-05-11T01:01:45.293477+0000","last_became_peered":"2024-05-11T01:01:45.293477+0000","last_unstale":"2024-05-11T01:01:45.298813+0000","last_undegraded":"2024-05-11T01:01:45.298813+0000","last_fullsized":"2024-05-11T01:01:45.298813+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:12:42.551695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.292147+0000","last_change":"2024-05-11T01:01:45.290112+0000","last_active":"2024-05-11T01:01:45.292147+0000","last_peered":"2024-05-11T01:01:45.292147+0000","last_clean":"2024-05-11T01:01:45.292147+0000","last_became_active":"2024-05-11T01:01:45.289808+0000","last_became_peered":"2024-05-11T01:01:45.289808+0000","last_unstale":"2024-05-11T01:01:45.292147+0000","last_undegraded":"2024-05-11T01:01:45.292147+0000","last_fullsized":"2024-05-11T01:01:45.292147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:24.197518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.288384+0000","last_change":"2024-05-11T01:01:45.285375+0000","last_active":"2024-05-11T01:01:45.288384+0000","last_peered":"2024-05-11T01:01:45.288384+0000","last_clean":"2024-05-11T01:01:45.288384+0000","last_became_active":"2024-05-11T01:01:45.284943+0000","last_became_peered":"2024-05-11T01:01:45.284943+0000","last_unstale":"2024-05-11T01:01:45.288384+0000","last_undegraded":"2024-05-11T01:01:45.288384+0000","last_fullsized":"2024-05-11T01:01:45.288384+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:02:37.642922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.304077+0000","last_change":"2024-05-11T01:01:45.300881+0000","last_active":"2024-05-11T01:01:45.304077+0000","last_peered":"2024-05-11T01:01:45.304077+0000","last_clean":"2024-05-11T01:01:45.304077+0000","last_became_active":"2024-05-11T01:01:45.299898+0000","last_became_peered":"2024-05-11T01:01:45.299898+0000","last_unstale":"2024-05-11T01:01:45.304077+0000","last_undegraded":"2024-05-11T01:01:45.304077+0000","last_fullsized":"2024-05-11T01:01:45.304077+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:12:42.551695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289239+0000","last_change":"2024-05-11T01:01:45.288279+0000","last_active":"2024-05-11T01:01:45.289239+0000","last_peered":"2024-05-11T01:01:45.289239+0000","last_clean":"2024-05-11T01:01:45.289239+0000","last_became_active":"2024-05-11T01:01:45.287969+0000","last_became_peered":"2024-05-11T01:01:45.287969+0000","last_unstale":"2024-05-11T01:01:45.289239+0000","last_undegraded":"2024-05-11T01:01:45.289239+0000","last_fullsized":"2024-05-11T01:01:45.289239+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:40.047660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.292803+0000","last_change":"2024-05-11T01:01:45.286582+0000","last_active":"2024-05-11T01:01:45.292803+0000","last_peered":"2024-05-11T01:01:45.292803+0000","last_clean":"2024-05-11T01:01:45.292803+0000","last_became_active":"2024-05-11T01:01:45.286338+0000","last_became_peered":"2024-05-11T01:01:45.286338+0000","last_unstale":"2024-05-11T01:01:45.292803+0000","last_undegraded":"2024-05-11T01:01:45.292803+0000","last_fullsized":"2024-05-11T01:01:45.292803+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:17.182079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301390+0000","last_change":"2024-05-11T01:01:45.290183+0000","last_active":"2024-05-11T01:01:45.301390+0000","last_peered":"2024-05-11T01:01:45.301390+0000","last_clean":"2024-05-11T01:01:45.301390+0000","last_became_active":"2024-05-11T01:01:45.288298+0000","last_became_peered":"2024-05-11T01:01:45.288298+0000","last_unstale":"2024-05-11T01:01:45.301390+0000","last_undegraded":"2024-05-11T01:01:45.301390+0000","last_fullsized":"2024-05-11T01:01:45.301390+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:14:32.666306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305816+0000","last_change":"2024-05-11T01:01:45.304408+0000","last_active":"2024-05-11T01:01:45.305816+0000","last_peered":"2024-05-11T01:01:45.305816+0000","last_clean":"2024-05-11T01:01:45.305816+0000","last_became_active":"2024-05-11T01:01:45.304125+0000","last_became_peered":"2024-05-11T01:01:45.304125+0000","last_unstale":"2024-05-11T01:01:45.305816+0000","last_undegraded":"2024-05-11T01:01:45.305816+0000","last_fullsized":"2024-05-11T01:01:45.305816+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:33:40.871894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.292295+0000","last_change":"2024-05-11T01:01:45.287732+0000","last_active":"2024-05-11T01:01:45.292295+0000","last_peered":"2024-05-11T01:01:45.292295+0000","last_clean":"2024-05-11T01:01:45.292295+0000","last_became_active":"2024-05-11T01:01:45.287438+0000","last_became_peered":"2024-05-11T01:01:45.287438+0000","last_unstale":"2024-05-11T01:01:45.292295+0000","last_undegraded":"2024-05-11T01:01:45.292295+0000","last_fullsized":"2024-05-11T01:01:45.292295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:14.953184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.306374+0000","last_change":"2024-05-11T01:01:45.300595+0000","last_active":"2024-05-11T01:01:45.306374+0000","last_peered":"2024-05-11T01:01:45.306374+0000","last_clean":"2024-05-11T01:01:45.306374+0000","last_became_active":"2024-05-11T01:01:45.300361+0000","last_became_peered":"2024-05-11T01:01:45.300361+0000","last_unstale":"2024-05-11T01:01:45.306374+0000","last_undegraded":"2024-05-11T01:01:45.306374+0000","last_fullsized":"2024-05-11T01:01:45.306374+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:07:10.746755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.294100+0000","last_change":"2024-05-11T01:01:45.290986+0000","last_active":"2024-05-11T01:01:45.294100+0000","last_peered":"2024-05-11T01:01:45.294100+0000","last_clean":"2024-05-11T01:01:45.294100+0000","last_became_active":"2024-05-11T01:01:45.290579+0000","last_became_peered":"2024-05-11T01:01:45.290579+0000","last_unstale":"2024-05-11T01:01:45.294100+0000","last_undegraded":"2024-05-11T01:01:45.294100+0000","last_fullsized":"2024-05-11T01:01:45.294100+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.295161+0000","last_change":"2024-05-11T01:01:45.284671+0000","last_active":"2024-05-11T01:01:45.295161+0000","last_peered":"2024-05-11T01:01:45.295161+0000","last_clean":"2024-05-11T01:01:45.295161+0000","last_became_active":"2024-05-11T01:01:45.283784+0000","last_became_peered":"2024-05-11T01:01:45.283784+0000","last_unstale":"2024-05-11T01:01:45.295161+0000","last_undegraded":"2024-05-11T01:01:45.295161+0000","last_fullsized":"2024-05-11T01:01:45.295161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.300516+0000","last_change":"2024-05-11T01:01:45.296460+0000","last_active":"2024-05-11T01:01:45.300516+0000","last_peered":"2024-05-11T01:01:45.300516+0000","last_clean":"2024-05-11T01:01:45.300516+0000","last_became_active":"2024-05-11T01:01:45.296154+0000","last_became_peered":"2024-05-11T01:01:45.296154+0000","last_unstale":"2024-05-11T01:01:45.300516+0000","last_undegraded":"2024-05-11T01:01:45.300516+0000","last_fullsized":"2024-05-11T01:01:45.300516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:49:08.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301303+0000","last_change":"2024-05-11T01:01:45.293455+0000","last_active":"2024-05-11T01:01:45.301303+0000","last_peered":"2024-05-11T01:01:45.301303+0000","last_clean":"2024-05-11T01:01:45.301303+0000","last_became_active":"2024-05-11T01:01:45.293133+0000","last_became_peered":"2024-05-11T01:01:45.293133+0000","last_unstale":"2024-05-11T01:01:45.301303+0000","last_undegraded":"2024-05-11T01:01:45.301303+0000","last_fullsized":"2024-05-11T01:01:45.301303+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:53.959890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.297957+0000","last_change":"2024-05-11T01:01:45.290392+0000","last_active":"2024-05-11T01:01:45.297957+0000","last_peered":"2024-05-11T01:01:45.297957+0000","last_clean":"2024-05-11T01:01:45.297957+0000","last_became_active":"2024-05-11T01:01:45.290160+0000","last_became_peered":"2024-05-11T01:01:45.290160+0000","last_unstale":"2024-05-11T01:01:45.297957+0000","last_undegraded":"2024-05-11T01:01:45.297957+0000","last_fullsized":"2024-05-11T01:01:45.297957+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:24.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299305+0000","last_change":"2024-05-11T01:01:45.297443+0000","last_active":"2024-05-11T01:01:45.299305+0000","last_peered":"2024-05-11T01:01:45.299305+0000","last_clean":"2024-05-11T01:01:45.299305+0000","last_became_active":"2024-05-11T01:01:45.296817+0000","last_became_peered":"2024-05-11T01:01:45.296817+0000","last_unstale":"2024-05-11T01:01:45.299305+0000","last_undegraded":"2024-05-11T01:01:45.299305+0000","last_fullsized":"2024-05-11T01:01:45.299305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:58.923752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.303498+0000","last_change":"2024-05-11T01:01:45.302563+0000","last_active":"2024-05-11T01:01:45.303498+0000","last_peered":"2024-05-11T01:01:45.303498+0000","last_clean":"2024-05-11T01:01:45.303498+0000","last_became_active":"2024-05-11T01:01:45.302249+0000","last_became_peered":"2024-05-11T01:01:45.302249+0000","last_unstale":"2024-05-11T01:01:45.303498+0000","last_undegraded":"2024-05-11T01:01:45.303498+0000","last_fullsized":"2024-05-11T01:01:45.303498+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:17:11.351660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289737+0000","last_change":"2024-05-11T01:01:45.282806+0000","last_active":"2024-05-11T01:01:45.289737+0000","last_peered":"2024-05-11T01:01:45.289737+0000","last_clean":"2024-05-11T01:01:45.289737+0000","last_became_active":"2024-05-11T01:01:45.282583+0000","last_became_peered":"2024-05-11T01:01:45.282583+0000","last_unstale":"2024-05-11T01:01:45.289737+0000","last_undegraded":"2024-05-11T01:01:45.289737+0000","last_fullsized":"2024-05-11T01:01:45.289737+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.296508+0000","last_change":"2024-05-11T01:01:45.291086+0000","last_active":"2024-05-11T01:01:45.296508+0000","last_peered":"2024-05-11T01:01:45.296508+0000","last_clean":"2024-05-11T01:01:45.296508+0000","last_became_active":"2024-05-11T01:01:45.290335+0000","last_became_peered":"2024-05-11T01:01:45.290335+0000","last_unstale":"2024-05-11T01:01:45.296508+0000","last_undegraded":"2024-05-11T01:01:45.296508+0000","last_fullsized":"2024-05-11T01:01:45.296508+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:33.401596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.304678+0000","last_change":"2024-05-11T01:01:45.296462+0000","last_active":"2024-05-11T01:01:45.304678+0000","last_peered":"2024-05-11T01:01:45.304678+0000","last_clean":"2024-05-11T01:01:45.304678+0000","last_became_active":"2024-05-11T01:01:45.294415+0000","last_became_peered":"2024-05-11T01:01:45.294415+0000","last_unstale":"2024-05-11T01:01:45.304678+0000","last_undegraded":"2024-05-11T01:01:45.304678+0000","last_fullsized":"2024-05-11T01:01:45.304678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:07:10.746755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.286402+0000","last_change":"2024-05-11T01:01:45.282837+0000","last_active":"2024-05-11T01:01:45.286402+0000","last_peered":"2024-05-11T01:01:45.286402+0000","last_clean":"2024-05-11T01:01:45.286402+0000","last_became_active":"2024-05-11T01:01:45.281896+0000","last_became_peered":"2024-05-11T01:01:45.281896+0000","last_unstale":"2024-05-11T01:01:45.286402+0000","last_undegraded":"2024-05-11T01:01:45.286402+0000","last_fullsized":"2024-05-11T01:01:45.286402+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:58.923752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299271+0000","last_change":"2024-05-11T01:01:45.292996+0000","last_active":"2024-05-11T01:01:45.299271+0000","last_peered":"2024-05-11T01:01:45.299271+0000","last_clean":"2024-05-11T01:01:45.299271+0000","last_became_active":"2024-05-11T01:01:45.292623+0000","last_became_peered":"2024-05-11T01:01:45.292623+0000","last_unstale":"2024-05-11T01:01:45.299271+0000","last_undegraded":"2024-05-11T01:01:45.299271+0000","last_fullsized":"2024-05-11T01:01:45.299271+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:49:08.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299050+0000","last_change":"2024-05-11T01:01:45.297337+0000","last_active":"2024-05-11T01:01:45.299050+0000","last_peered":"2024-05-11T01:01:45.299050+0000","last_clean":"2024-05-11T01:01:45.299050+0000","last_became_active":"2024-05-11T01:01:45.297031+0000","last_became_peered":"2024-05-11T01:01:45.297031+0000","last_unstale":"2024-05-11T01:01:45.299050+0000","last_undegraded":"2024-05-11T01:01:45.299050+0000","last_fullsized":"2024-05-11T01:01:45.299050+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:06.518581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289215+0000","last_change":"2024-05-11T01:01:45.288295+0000","last_active":"2024-05-11T01:01:45.289215+0000","last_peered":"2024-05-11T01:01:45.289215+0000","last_clean":"2024-05-11T01:01:45.289215+0000","last_became_active":"2024-05-11T01:01:45.288000+0000","last_became_peered":"2024-05-11T01:01:45.288000+0000","last_unstale":"2024-05-11T01:01:45.289215+0000","last_undegraded":"2024-05-11T01:01:45.289215+0000","last_fullsized":"2024-05-11T01:01:45.289215+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:33.401596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308034+0000","last_change":"2024-05-11T01:01:45.290840+0000","last_active":"2024-05-11T01:01:46.308034+0000","last_peered":"2024-05-11T01:01:46.308034+0000","last_clean":"2024-05-11T01:01:46.308034+0000","last_became_active":"2024-05-11T01:01:45.290214+0000","last_became_peered":"2024-05-11T01:01:45.290214+0000","last_unstale":"2024-05-11T01:01:46.308034+0000","last_undegraded":"2024-05-11T01:01:46.308034+0000","last_fullsized":"2024-05-11T01:01:46.308034+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:58.923752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299310+0000","last_change":"2024-05-11T01:01:45.297105+0000","last_active":"2024-05-11T01:01:45.299310+0000","last_peered":"2024-05-11T01:01:45.299310+0000","last_clean":"2024-05-11T01:01:45.299310+0000","last_became_active":"2024-05-11T01:01:45.296786+0000","last_became_peered":"2024-05-11T01:01:45.296786+0000","last_unstale":"2024-05-11T01:01:45.299310+0000","last_undegraded":"2024-05-11T01:01:45.299310+0000","last_fullsized":"2024-05-11T01:01:45.299310+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:17.182079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.302763+0000","last_change":"2024-05-11T01:01:45.301925+0000","last_active":"2024-05-11T01:01:45.302763+0000","last_peered":"2024-05-11T01:01:45.302763+0000","last_clean":"2024-05-11T01:01:45.302763+0000","last_became_active":"2024-05-11T01:01:45.301643+0000","last_became_peered":"2024-05-11T01:01:45.301643+0000","last_unstale":"2024-05-11T01:01:45.302763+0000","last_undegraded":"2024-05-11T01:01:45.302763+0000","last_fullsized":"2024-05-11T01:01:45.302763+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:02:37.642922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308657+0000","last_change":"2024-05-11T01:01:44.285791+0000","last_active":"2024-05-11T01:01:46.308657+0000","last_peered":"2024-05-11T01:01:46.308657+0000","last_clean":"2024-05-11T01:01:46.308657+0000","last_became_active":"2024-05-11T01:01:44.285482+0000","last_became_peered":"2024-05-11T01:01:44.285482+0000","last_unstale":"2024-05-11T01:01:46.308657+0000","last_undegraded":"2024-05-11T01:01:46.308657+0000","last_fullsized":"2024-05-11T01:01:46.308657+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:50.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308555+0000","last_change":"2024-05-11T01:01:45.295694+0000","last_active":"2024-05-11T01:01:46.308555+0000","last_peered":"2024-05-11T01:01:46.308555+0000","last_clean":"2024-05-11T01:01:46.308555+0000","last_became_active":"2024-05-11T01:01:45.295432+0000","last_became_peered":"2024-05-11T01:01:45.295432+0000","last_unstale":"2024-05-11T01:01:46.308555+0000","last_undegraded":"2024-05-11T01:01:46.308555+0000","last_fullsized":"2024-05-11T01:01:46.308555+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.307288+0000","last_change":"2024-05-11T01:01:44.286162+0000","last_active":"2024-05-11T01:01:45.307288+0000","last_peered":"2024-05-11T01:01:45.307288+0000","last_clean":"2024-05-11T01:01:45.307288+0000","last_became_active":"2024-05-11T01:01:44.285818+0000","last_became_peered":"2024-05-11T01:01:44.285818+0000","last_unstale":"2024-05-11T01:01:45.307288+0000","last_undegraded":"2024-05-11T01:01:45.307288+0000","last_fullsized":"2024-05-11T01:01:45.307288+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:21:20.490794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305604+0000","last_change":"2024-05-11T01:01:45.304646+0000","last_active":"2024-05-11T01:01:45.305604+0000","last_peered":"2024-05-11T01:01:45.305604+0000","last_clean":"2024-05-11T01:01:45.305604+0000","last_became_active":"2024-05-11T01:01:45.304404+0000","last_became_peered":"2024-05-11T01:01:45.304404+0000","last_unstale":"2024-05-11T01:01:45.305604+0000","last_undegraded":"2024-05-11T01:01:45.305604+0000","last_fullsized":"2024-05-11T01:01:45.305604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:40.047660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.304029+0000","last_change":"2024-05-11T01:01:44.290314+0000","last_active":"2024-05-11T01:01:45.304029+0000","last_peered":"2024-05-11T01:01:45.304029+0000","last_clean":"2024-05-11T01:01:45.304029+0000","last_became_active":"2024-05-11T01:01:44.290042+0000","last_became_peered":"2024-05-11T01:01:44.290042+0000","last_unstale":"2024-05-11T01:01:45.304029+0000","last_undegraded":"2024-05-11T01:01:45.304029+0000","last_fullsized":"2024-05-11T01:01:45.304029+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:05.512999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.286425+0000","last_change":"2024-05-11T01:01:45.282871+0000","last_active":"2024-05-11T01:01:45.286425+0000","last_peered":"2024-05-11T01:01:45.286425+0000","last_clean":"2024-05-11T01:01:45.286425+0000","last_became_active":"2024-05-11T01:01:45.281469+0000","last_became_peered":"2024-05-11T01:01:45.281469+0000","last_unstale":"2024-05-11T01:01:45.286425+0000","last_undegraded":"2024-05-11T01:01:45.286425+0000","last_fullsized":"2024-05-11T01:01:45.286425+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:40.047660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.307337+0000","last_change":"2024-05-11T01:01:44.291756+0000","last_active":"2024-05-11T01:01:45.307337+0000","last_peered":"2024-05-11T01:01:45.307337+0000","last_clean":"2024-05-11T01:01:45.307337+0000","last_became_active":"2024-05-11T01:01:44.291428+0000","last_became_peered":"2024-05-11T01:01:44.291428+0000","last_unstale":"2024-05-11T01:01:45.307337+0000","last_undegraded":"2024-05-11T01:01:45.307337+0000","last_fullsized":"2024-05-11T01:01:45.307337+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:37:15.659807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.306324+0000","last_change":"2024-05-11T01:01:45.298629+0000","last_active":"2024-05-11T01:01:45.306324+0000","last_peered":"2024-05-11T01:01:45.306324+0000","last_clean":"2024-05-11T01:01:45.306324+0000","last_became_active":"2024-05-11T01:01:45.298312+0000","last_became_peered":"2024-05-11T01:01:45.298312+0000","last_unstale":"2024-05-11T01:01:45.306324+0000","last_undegraded":"2024-05-11T01:01:45.306324+0000","last_fullsized":"2024-05-11T01:01:45.306324+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:14.953184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305837+0000","last_change":"2024-05-11T01:01:44.288609+0000","last_active":"2024-05-11T01:01:45.305837+0000","last_peered":"2024-05-11T01:01:45.305837+0000","last_clean":"2024-05-11T01:01:45.305837+0000","last_became_active":"2024-05-11T01:01:44.288308+0000","last_became_peered":"2024-05-11T01:01:44.288308+0000","last_unstale":"2024-05-11T01:01:45.305837+0000","last_undegraded":"2024-05-11T01:01:45.305837+0000","last_fullsized":"2024-05-11T01:01:45.305837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:17:19.132665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299021+0000","last_change":"2024-05-11T01:01:45.293175+0000","last_active":"2024-05-11T01:01:45.299021+0000","last_peered":"2024-05-11T01:01:45.299021+0000","last_clean":"2024-05-11T01:01:45.299021+0000","last_became_active":"2024-05-11T01:01:45.292801+0000","last_became_peered":"2024-05-11T01:01:45.292801+0000","last_unstale":"2024-05-11T01:01:45.299021+0000","last_undegraded":"2024-05-11T01:01:45.299021+0000","last_fullsized":"2024-05-11T01:01:45.299021+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:07:10.746755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305828+0000","last_change":"2024-05-11T01:01:44.287857+0000","last_active":"2024-05-11T01:01:45.305828+0000","last_peered":"2024-05-11T01:01:45.305828+0000","last_clean":"2024-05-11T01:01:45.305828+0000","last_became_active":"2024-05-11T01:01:44.287580+0000","last_became_peered":"2024-05-11T01:01:44.287580+0000","last_unstale":"2024-05-11T01:01:45.305828+0000","last_undegraded":"2024-05-11T01:01:45.305828+0000","last_fullsized":"2024-05-11T01:01:45.305828+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:00:46.594424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.298037+0000","last_change":"2024-05-11T01:01:45.289642+0000","last_active":"2024-05-11T01:01:45.298037+0000","last_peered":"2024-05-11T01:01:45.298037+0000","last_clean":"2024-05-11T01:01:45.298037+0000","last_became_active":"2024-05-11T01:01:45.289324+0000","last_became_peered":"2024-05-11T01:01:45.289324+0000","last_unstale":"2024-05-11T01:01:45.298037+0000","last_undegraded":"2024-05-11T01:01:45.298037+0000","last_fullsized":"2024-05-11T01:01:45.298037+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:17:11.351660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.275039+0000","last_change":"2024-05-11T01:01:44.280527+0000","last_active":"2024-05-11T01:01:45.275039+0000","last_peered":"2024-05-11T01:01:45.275039+0000","last_clean":"2024-05-11T01:01:45.275039+0000","last_became_active":"2024-05-11T01:01:44.280236+0000","last_became_peered":"2024-05-11T01:01:44.280236+0000","last_unstale":"2024-05-11T01:01:45.275039+0000","last_undegraded":"2024-05-11T01:01:45.275039+0000","last_fullsized":"2024-05-11T01:01:45.275039+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:43:00.679498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.297355+0000","last_change":"2024-05-11T01:01:45.293282+0000","last_active":"2024-05-11T01:01:45.297355+0000","last_peered":"2024-05-11T01:01:45.297355+0000","last_clean":"2024-05-11T01:01:45.297355+0000","last_became_active":"2024-05-11T01:01:45.292766+0000","last_became_peered":"2024-05-11T01:01:45.292766+0000","last_unstale":"2024-05-11T01:01:45.297355+0000","last_undegraded":"2024-05-11T01:01:45.297355+0000","last_fullsized":"2024-05-11T01:01:45.297355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:11:53.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.278251+0000","last_change":"2024-05-11T01:01:44.285917+0000","last_active":"2024-05-11T01:01:45.278251+0000","last_peered":"2024-05-11T01:01:45.278251+0000","last_clean":"2024-05-11T01:01:45.278251+0000","last_became_active":"2024-05-11T01:01:44.285631+0000","last_became_peered":"2024-05-11T01:01:44.285631+0000","last_unstale":"2024-05-11T01:01:45.278251+0000","last_undegraded":"2024-05-11T01:01:45.278251+0000","last_fullsized":"2024-05-11T01:01:45.278251+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:52.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289468+0000","last_change":"2024-05-11T01:01:45.282249+0000","last_active":"2024-05-11T01:01:45.289468+0000","last_peered":"2024-05-11T01:01:45.289468+0000","last_clean":"2024-05-11T01:01:45.289468+0000","last_became_active":"2024-05-11T01:01:45.281956+0000","last_became_peered":"2024-05-11T01:01:45.281956+0000","last_unstale":"2024-05-11T01:01:45.289468+0000","last_undegraded":"2024-05-11T01:01:45.289468+0000","last_fullsized":"2024-05-11T01:01:45.289468+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:24.197518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.276109+0000","last_change":"2024-05-11T01:01:44.285505+0000","last_active":"2024-05-11T01:01:45.276109+0000","last_peered":"2024-05-11T01:01:45.276109+0000","last_clean":"2024-05-11T01:01:45.276109+0000","last_became_active":"2024-05-11T01:01:44.285240+0000","last_became_peered":"2024-05-11T01:01:44.285240+0000","last_unstale":"2024-05-11T01:01:45.276109+0000","last_undegraded":"2024-05-11T01:01:45.276109+0000","last_fullsized":"2024-05-11T01:01:45.276109+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:57.918170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.300483+0000","last_change":"2024-05-11T01:01:45.295564+0000","last_active":"2024-05-11T01:01:45.300483+0000","last_peered":"2024-05-11T01:01:45.300483+0000","last_clean":"2024-05-11T01:01:45.300483+0000","last_became_active":"2024-05-11T01:01:45.295237+0000","last_became_peered":"2024-05-11T01:01:45.295237+0000","last_unstale":"2024-05-11T01:01:45.300483+0000","last_undegraded":"2024-05-11T01:01:45.300483+0000","last_fullsized":"2024-05-11T01:01:45.300483+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:17.182079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.283602+0000","last_change":"2024-05-11T01:01:44.280751+0000","last_active":"2024-05-11T01:01:45.283602+0000","last_peered":"2024-05-11T01:01:45.283602+0000","last_clean":"2024-05-11T01:01:45.283602+0000","last_became_active":"2024-05-11T01:01:44.280479+0000","last_became_peered":"2024-05-11T01:01:44.280479+0000","last_unstale":"2024-05-11T01:01:45.283602+0000","last_undegraded":"2024-05-11T01:01:45.283602+0000","last_fullsized":"2024-05-11T01:01:45.283602+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:45:35.410817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.297604+0000","last_change":"2024-05-11T01:01:45.294751+0000","last_active":"2024-05-11T01:01:45.297604+0000","last_peered":"2024-05-11T01:01:45.297604+0000","last_clean":"2024-05-11T01:01:45.297604+0000","last_became_active":"2024-05-11T01:01:45.294429+0000","last_became_peered":"2024-05-11T01:01:45.294429+0000","last_unstale":"2024-05-11T01:01:45.297604+0000","last_undegraded":"2024-05-11T01:01:45.297604+0000","last_fullsized":"2024-05-11T01:01:45.297604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:23:44.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.296428+0000","last_change":"2024-05-11T01:01:44.287411+0000","last_active":"2024-05-11T01:01:45.296428+0000","last_peered":"2024-05-11T01:01:45.296428+0000","last_clean":"2024-05-11T01:01:45.296428+0000","last_became_active":"2024-05-11T01:01:44.286927+0000","last_became_peered":"2024-05-11T01:01:44.286927+0000","last_unstale":"2024-05-11T01:01:45.296428+0000","last_undegraded":"2024-05-11T01:01:45.296428+0000","last_fullsized":"2024-05-11T01:01:45.296428+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:57.918170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289919+0000","last_change":"2024-05-11T01:01:45.286021+0000","last_active":"2024-05-11T01:01:45.289919+0000","last_peered":"2024-05-11T01:01:45.289919+0000","last_clean":"2024-05-11T01:01:45.289919+0000","last_became_active":"2024-05-11T01:01:45.285721+0000","last_became_peered":"2024-05-11T01:01:45.285721+0000","last_unstale":"2024-05-11T01:01:45.289919+0000","last_undegraded":"2024-05-11T01:01:45.289919+0000","last_fullsized":"2024-05-11T01:01:45.289919+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:12:42.551695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.308523+0000","last_change":"2024-05-11T01:01:44.289402+0000","last_active":"2024-05-11T01:01:45.308523+0000","last_peered":"2024-05-11T01:01:45.308523+0000","last_clean":"2024-05-11T01:01:45.308523+0000","last_became_active":"2024-05-11T01:01:44.289115+0000","last_became_peered":"2024-05-11T01:01:44.289115+0000","last_unstale":"2024-05-11T01:01:45.308523+0000","last_undegraded":"2024-05-11T01:01:45.308523+0000","last_fullsized":"2024-05-11T01:01:45.308523+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:32.396014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.297394+0000","last_change":"2024-05-11T01:01:45.293221+0000","last_active":"2024-05-11T01:01:45.297394+0000","last_peered":"2024-05-11T01:01:45.297394+0000","last_clean":"2024-05-11T01:01:45.297394+0000","last_became_active":"2024-05-11T01:01:45.292910+0000","last_became_peered":"2024-05-11T01:01:45.292910+0000","last_unstale":"2024-05-11T01:01:45.297394+0000","last_undegraded":"2024-05-11T01:01:45.297394+0000","last_fullsized":"2024-05-11T01:01:45.297394+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301103+0000","last_change":"2024-05-11T01:01:44.287627+0000","last_active":"2024-05-11T01:01:45.301103+0000","last_peered":"2024-05-11T01:01:45.301103+0000","last_clean":"2024-05-11T01:01:45.301103+0000","last_became_active":"2024-05-11T01:01:44.287334+0000","last_became_peered":"2024-05-11T01:01:44.287334+0000","last_unstale":"2024-05-11T01:01:45.301103+0000","last_undegraded":"2024-05-11T01:01:45.301103+0000","last_fullsized":"2024-05-11T01:01:45.301103+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:45:35.410817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.294838+0000","last_change":"2024-05-11T01:01:45.290907+0000","last_active":"2024-05-11T01:01:45.294838+0000","last_peered":"2024-05-11T01:01:45.294838+0000","last_clean":"2024-05-11T01:01:45.294838+0000","last_became_active":"2024-05-11T01:01:45.290606+0000","last_became_peered":"2024-05-11T01:01:45.290606+0000","last_unstale":"2024-05-11T01:01:45.294838+0000","last_undegraded":"2024-05-11T01:01:45.294838+0000","last_fullsized":"2024-05-11T01:01:45.294838+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:12:42.551695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.277510+0000","last_change":"2024-05-11T01:01:44.289402+0000","last_active":"2024-05-11T01:01:45.277510+0000","last_peered":"2024-05-11T01:01:45.277510+0000","last_clean":"2024-05-11T01:01:45.277510+0000","last_became_active":"2024-05-11T01:01:44.289138+0000","last_became_peered":"2024-05-11T01:01:44.289138+0000","last_unstale":"2024-05-11T01:01:45.277510+0000","last_undegraded":"2024-05-11T01:01:45.277510+0000","last_fullsized":"2024-05-11T01:01:45.277510+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:17:10.346079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.306352+0000","last_change":"2024-05-11T01:01:45.299228+0000","last_active":"2024-05-11T01:01:45.306352+0000","last_peered":"2024-05-11T01:01:45.306352+0000","last_clean":"2024-05-11T01:01:45.306352+0000","last_became_active":"2024-05-11T01:01:45.299013+0000","last_became_peered":"2024-05-11T01:01:45.299013+0000","last_unstale":"2024-05-11T01:01:45.306352+0000","last_undegraded":"2024-05-11T01:01:45.306352+0000","last_fullsized":"2024-05-11T01:01:45.306352+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:49:08.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.274889+0000","last_change":"2024-05-11T01:01:44.285570+0000","last_active":"2024-05-11T01:01:45.274889+0000","last_peered":"2024-05-11T01:01:45.274889+0000","last_clean":"2024-05-11T01:01:45.274889+0000","last_became_active":"2024-05-11T01:01:44.285277+0000","last_became_peered":"2024-05-11T01:01:44.285277+0000","last_unstale":"2024-05-11T01:01:45.274889+0000","last_undegraded":"2024-05-11T01:01:45.274889+0000","last_fullsized":"2024-05-11T01:01:45.274889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:23.115690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308180+0000","last_change":"2024-05-11T01:01:45.295740+0000","last_active":"2024-05-11T01:01:46.308180+0000","last_peered":"2024-05-11T01:01:46.308180+0000","last_clean":"2024-05-11T01:01:46.308180+0000","last_became_active":"2024-05-11T01:01:45.295233+0000","last_became_peered":"2024-05-11T01:01:45.295233+0000","last_unstale":"2024-05-11T01:01:46.308180+0000","last_undegraded":"2024-05-11T01:01:46.308180+0000","last_fullsized":"2024-05-11T01:01:46.308180+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:33.401596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305543+0000","last_change":"2024-05-11T01:01:44.284493+0000","last_active":"2024-05-11T01:01:45.305543+0000","last_peered":"2024-05-11T01:01:45.305543+0000","last_clean":"2024-05-11T01:01:45.305543+0000","last_became_active":"2024-05-11T01:01:44.284192+0000","last_became_peered":"2024-05-11T01:01:44.284192+0000","last_unstale":"2024-05-11T01:01:45.305543+0000","last_undegraded":"2024-05-11T01:01:45.305543+0000","last_fullsized":"2024-05-11T01:01:45.305543+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:23.191937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.283182+0000","last_change":"2024-05-11T01:01:45.280417+0000","last_active":"2024-05-11T01:01:45.283182+0000","last_peered":"2024-05-11T01:01:45.283182+0000","last_clean":"2024-05-11T01:01:45.283182+0000","last_became_active":"2024-05-11T01:01:45.280111+0000","last_became_peered":"2024-05-11T01:01:45.280111+0000","last_unstale":"2024-05-11T01:01:45.283182+0000","last_undegraded":"2024-05-11T01:01:45.283182+0000","last_fullsized":"2024-05-11T01:01:45.283182+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:51.236037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.306100+0000","last_change":"2024-05-11T01:01:44.288373+0000","last_active":"2024-05-11T01:01:45.306100+0000","last_peered":"2024-05-11T01:01:45.306100+0000","last_clean":"2024-05-11T01:01:45.306100+0000","last_became_active":"2024-05-11T01:01:44.288055+0000","last_became_peered":"2024-05-11T01:01:44.288055+0000","last_unstale":"2024-05-11T01:01:45.306100+0000","last_undegraded":"2024-05-11T01:01:45.306100+0000","last_fullsized":"2024-05-11T01:01:45.306100+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:50.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.288866+0000","last_change":"2024-05-11T01:01:45.281226+0000","last_active":"2024-05-11T01:01:45.288866+0000","last_peered":"2024-05-11T01:01:45.288866+0000","last_clean":"2024-05-11T01:01:45.288866+0000","last_became_active":"2024-05-11T01:01:45.280950+0000","last_became_peered":"2024-05-11T01:01:45.280950+0000","last_unstale":"2024-05-11T01:01:45.288866+0000","last_undegraded":"2024-05-11T01:01:45.288866+0000","last_fullsized":"2024-05-11T01:01:45.288866+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:33.401596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.298906+0000","last_change":"2024-05-11T01:01:44.285957+0000","last_active":"2024-05-11T01:01:45.298906+0000","last_peered":"2024-05-11T01:01:45.298906+0000","last_clean":"2024-05-11T01:01:45.298906+0000","last_became_active":"2024-05-11T01:01:44.285672+0000","last_became_peered":"2024-05-11T01:01:44.285672+0000","last_unstale":"2024-05-11T01:01:45.298906+0000","last_undegraded":"2024-05-11T01:01:45.298906+0000","last_fullsized":"2024-05-11T01:01:45.298906+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:16.693227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.282633+0000","last_change":"2024-05-11T01:01:45.281528+0000","last_active":"2024-05-11T01:01:45.282633+0000","last_peered":"2024-05-11T01:01:45.282633+0000","last_clean":"2024-05-11T01:01:45.282633+0000","last_became_active":"2024-05-11T01:01:45.281191+0000","last_became_peered":"2024-05-11T01:01:45.281191+0000","last_unstale":"2024-05-11T01:01:45.282633+0000","last_undegraded":"2024-05-11T01:01:45.282633+0000","last_fullsized":"2024-05-11T01:01:45.282633+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:06.518581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301664+0000","last_change":"2024-05-11T01:01:44.281083+0000","last_active":"2024-05-11T01:01:45.301664+0000","last_peered":"2024-05-11T01:01:45.301664+0000","last_clean":"2024-05-11T01:01:45.301664+0000","last_became_active":"2024-05-11T01:01:44.280737+0000","last_became_peered":"2024-05-11T01:01:44.280737+0000","last_unstale":"2024-05-11T01:01:45.301664+0000","last_undegraded":"2024-05-11T01:01:45.301664+0000","last_fullsized":"2024-05-11T01:01:45.301664+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:24:37.529966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301438+0000","last_change":"2024-05-11T01:01:45.289836+0000","last_active":"2024-05-11T01:01:45.301438+0000","last_peered":"2024-05-11T01:01:45.301438+0000","last_clean":"2024-05-11T01:01:45.301438+0000","last_became_active":"2024-05-11T01:01:45.289511+0000","last_became_peered":"2024-05-11T01:01:45.289511+0000","last_unstale":"2024-05-11T01:01:45.301438+0000","last_undegraded":"2024-05-11T01:01:45.301438+0000","last_fullsized":"2024-05-11T01:01:45.301438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:36:37.828556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.278480+0000","last_change":"2024-05-11T01:01:44.288592+0000","last_active":"2024-05-11T01:01:45.278480+0000","last_peered":"2024-05-11T01:01:45.278480+0000","last_clean":"2024-05-11T01:01:45.278480+0000","last_became_active":"2024-05-11T01:01:44.288275+0000","last_became_peered":"2024-05-11T01:01:44.288275+0000","last_unstale":"2024-05-11T01:01:45.278480+0000","last_undegraded":"2024-05-11T01:01:45.278480+0000","last_fullsized":"2024-05-11T01:01:45.278480+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:23:10.377267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.288194+0000","last_change":"2024-05-11T01:01:45.282757+0000","last_active":"2024-05-11T01:01:45.288194+0000","last_peered":"2024-05-11T01:01:45.288194+0000","last_clean":"2024-05-11T01:01:45.288194+0000","last_became_active":"2024-05-11T01:01:45.281211+0000","last_became_peered":"2024-05-11T01:01:45.281211+0000","last_unstale":"2024-05-11T01:01:45.288194+0000","last_undegraded":"2024-05-11T01:01:45.288194+0000","last_fullsized":"2024-05-11T01:01:45.288194+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:27:08.460825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.307721+0000","last_change":"2024-05-11T01:01:44.288683+0000","last_active":"2024-05-11T01:01:45.307721+0000","last_peered":"2024-05-11T01:01:45.307721+0000","last_clean":"2024-05-11T01:01:45.307721+0000","last_became_active":"2024-05-11T01:01:44.288379+0000","last_became_peered":"2024-05-11T01:01:44.288379+0000","last_unstale":"2024-05-11T01:01:45.307721+0000","last_undegraded":"2024-05-11T01:01:45.307721+0000","last_fullsized":"2024-05-11T01:01:45.307721+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:24:37.529966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301370+0000","last_change":"2024-05-11T01:01:45.290037+0000","last_active":"2024-05-11T01:01:45.301370+0000","last_peered":"2024-05-11T01:01:45.301370+0000","last_clean":"2024-05-11T01:01:45.301370+0000","last_became_active":"2024-05-11T01:01:45.289765+0000","last_became_peered":"2024-05-11T01:01:45.289765+0000","last_unstale":"2024-05-11T01:01:45.301370+0000","last_undegraded":"2024-05-11T01:01:45.301370+0000","last_fullsized":"2024-05-11T01:01:45.301370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:16.460528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308214+0000","last_change":"2024-05-11T01:01:44.291362+0000","last_active":"2024-05-11T01:01:46.308214+0000","last_peered":"2024-05-11T01:01:46.308214+0000","last_clean":"2024-05-11T01:01:46.308214+0000","last_became_active":"2024-05-11T01:01:44.291003+0000","last_became_peered":"2024-05-11T01:01:44.291003+0000","last_unstale":"2024-05-11T01:01:46.308214+0000","last_undegraded":"2024-05-11T01:01:46.308214+0000","last_fullsized":"2024-05-11T01:01:46.308214+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:24:37.529966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301156+0000","last_change":"2024-05-11T01:01:45.290031+0000","last_active":"2024-05-11T01:01:45.301156+0000","last_peered":"2024-05-11T01:01:45.301156+0000","last_clean":"2024-05-11T01:01:45.301156+0000","last_became_active":"2024-05-11T01:01:45.289719+0000","last_became_peered":"2024-05-11T01:01:45.289719+0000","last_unstale":"2024-05-11T01:01:45.301156+0000","last_undegraded":"2024-05-11T01:01:45.301156+0000","last_fullsized":"2024-05-11T01:01:45.301156+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:01.490189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.278256+0000","last_change":"2024-05-11T01:01:44.291340+0000","last_active":"2024-05-11T01:01:45.278256+0000","last_peered":"2024-05-11T01:01:45.278256+0000","last_clean":"2024-05-11T01:01:45.278256+0000","last_became_active":"2024-05-11T01:01:44.291028+0000","last_became_peered":"2024-05-11T01:01:44.291028+0000","last_unstale":"2024-05-11T01:01:45.278256+0000","last_undegraded":"2024-05-11T01:01:45.278256+0000","last_fullsized":"2024-05-11T01:01:45.278256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:55.593960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.295915+0000","last_change":"2024-05-11T01:01:45.291784+0000","last_active":"2024-05-11T01:01:45.295915+0000","last_peered":"2024-05-11T01:01:45.295915+0000","last_clean":"2024-05-11T01:01:45.295915+0000","last_became_active":"2024-05-11T01:01:45.291431+0000","last_became_peered":"2024-05-11T01:01:45.291431+0000","last_unstale":"2024-05-11T01:01:45.295915+0000","last_undegraded":"2024-05-11T01:01:45.295915+0000","last_fullsized":"2024-05-11T01:01:45.295915+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:11:53.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.274817+0000","last_change":"2024-05-11T01:01:44.287271+0000","last_active":"2024-05-11T01:01:45.274817+0000","last_peered":"2024-05-11T01:01:45.274817+0000","last_clean":"2024-05-11T01:01:45.274817+0000","last_became_active":"2024-05-11T01:01:44.286962+0000","last_became_peered":"2024-05-11T01:01:44.286962+0000","last_unstale":"2024-05-11T01:01:45.274817+0000","last_undegraded":"2024-05-11T01:01:45.274817+0000","last_fullsized":"2024-05-11T01:01:45.274817+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:39.042079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301336+0000","last_change":"2024-05-11T01:01:45.292682+0000","last_active":"2024-05-11T01:01:45.301336+0000","last_peered":"2024-05-11T01:01:45.301336+0000","last_clean":"2024-05-11T01:01:45.301336+0000","last_became_active":"2024-05-11T01:01:45.292395+0000","last_became_peered":"2024-05-11T01:01:45.292395+0000","last_unstale":"2024-05-11T01:01:45.301336+0000","last_undegraded":"2024-05-11T01:01:45.301336+0000","last_fullsized":"2024-05-11T01:01:45.301336+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:15.492453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301699+0000","last_change":"2024-05-11T01:01:44.291611+0000","last_active":"2024-05-11T01:01:45.301699+0000","last_peered":"2024-05-11T01:01:45.301699+0000","last_clean":"2024-05-11T01:01:45.301699+0000","last_became_active":"2024-05-11T01:01:44.291194+0000","last_became_peered":"2024-05-11T01:01:44.291194+0000","last_unstale":"2024-05-11T01:01:45.301699+0000","last_undegraded":"2024-05-11T01:01:45.301699+0000","last_fullsized":"2024-05-11T01:01:45.301699+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:23.115690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289806+0000","last_change":"2024-05-11T01:01:45.285159+0000","last_active":"2024-05-11T01:01:45.289806+0000","last_peered":"2024-05-11T01:01:45.289806+0000","last_clean":"2024-05-11T01:01:45.289806+0000","last_became_active":"2024-05-11T01:01:45.284418+0000","last_became_peered":"2024-05-11T01:01:45.284418+0000","last_unstale":"2024-05-11T01:01:45.289806+0000","last_undegraded":"2024-05-11T01:01:45.289806+0000","last_fullsized":"2024-05-11T01:01:45.289806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:11:53.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.283749+0000","last_change":"2024-05-11T01:01:44.288773+0000","last_active":"2024-05-11T01:01:45.283749+0000","last_peered":"2024-05-11T01:01:45.283749+0000","last_clean":"2024-05-11T01:01:45.283749+0000","last_became_active":"2024-05-11T01:01:44.288477+0000","last_became_peered":"2024-05-11T01:01:44.288477+0000","last_unstale":"2024-05-11T01:01:45.283749+0000","last_undegraded":"2024-05-11T01:01:45.283749+0000","last_fullsized":"2024-05-11T01:01:45.283749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:16.693227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.291744+0000","last_change":"2024-05-11T01:01:45.289314+0000","last_active":"2024-05-11T01:01:45.291744+0000","last_peered":"2024-05-11T01:01:45.291744+0000","last_clean":"2024-05-11T01:01:45.291744+0000","last_became_active":"2024-05-11T01:01:45.288540+0000","last_became_peered":"2024-05-11T01:01:45.288540+0000","last_unstale":"2024-05-11T01:01:45.291744+0000","last_undegraded":"2024-05-11T01:01:45.291744+0000","last_fullsized":"2024-05-11T01:01:45.291744+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:51.236037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.275334+0000","last_change":"2024-05-11T01:01:44.288644+0000","last_active":"2024-05-11T01:01:45.275334+0000","last_peered":"2024-05-11T01:01:45.275334+0000","last_clean":"2024-05-11T01:01:45.275334+0000","last_became_active":"2024-05-11T01:01:44.288315+0000","last_became_peered":"2024-05-11T01:01:44.288315+0000","last_unstale":"2024-05-11T01:01:45.275334+0000","last_undegraded":"2024-05-11T01:01:45.275334+0000","last_fullsized":"2024-05-11T01:01:45.275334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:02:36.637341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308113+0000","last_change":"2024-05-11T01:01:45.303836+0000","last_active":"2024-05-11T01:01:46.308113+0000","last_peered":"2024-05-11T01:01:46.308113+0000","last_clean":"2024-05-11T01:01:46.308113+0000","last_became_active":"2024-05-11T01:01:45.303521+0000","last_became_peered":"2024-05-11T01:01:45.303521+0000","last_unstale":"2024-05-11T01:01:46.308113+0000","last_undegraded":"2024-05-11T01:01:46.308113+0000","last_fullsized":"2024-05-11T01:01:46.308113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:40.047660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.285641+0000","last_change":"2024-05-11T01:01:44.284801+0000","last_active":"2024-05-11T01:01:45.285641+0000","last_peered":"2024-05-11T01:01:45.285641+0000","last_clean":"2024-05-11T01:01:45.285641+0000","last_became_active":"2024-05-11T01:01:44.284389+0000","last_became_peered":"2024-05-11T01:01:44.284389+0000","last_unstale":"2024-05-11T01:01:45.285641+0000","last_undegraded":"2024-05-11T01:01:45.285641+0000","last_fullsized":"2024-05-11T01:01:45.285641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:17:19.132665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305297+0000","last_change":"2024-05-11T01:01:45.302950+0000","last_active":"2024-05-11T01:01:45.305297+0000","last_peered":"2024-05-11T01:01:45.305297+0000","last_clean":"2024-05-11T01:01:45.305297+0000","last_became_active":"2024-05-11T01:01:45.302359+0000","last_became_peered":"2024-05-11T01:01:45.302359+0000","last_unstale":"2024-05-11T01:01:45.305297+0000","last_undegraded":"2024-05-11T01:01:45.305297+0000","last_fullsized":"2024-05-11T01:01:45.305297+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:24.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.277701+0000","last_change":"2024-05-11T01:01:44.290395+0000","last_active":"2024-05-11T01:01:45.277701+0000","last_peered":"2024-05-11T01:01:45.277701+0000","last_clean":"2024-05-11T01:01:45.277701+0000","last_became_active":"2024-05-11T01:01:44.289930+0000","last_became_peered":"2024-05-11T01:01:44.289930+0000","last_unstale":"2024-05-11T01:01:45.277701+0000","last_undegraded":"2024-05-11T01:01:45.277701+0000","last_fullsized":"2024-05-11T01:01:45.277701+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:43:00.679498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.300594+0000","last_change":"2024-05-11T01:01:45.295756+0000","last_active":"2024-05-11T01:01:45.300594+0000","last_peered":"2024-05-11T01:01:45.300594+0000","last_clean":"2024-05-11T01:01:45.300594+0000","last_became_active":"2024-05-11T01:01:45.295261+0000","last_became_peered":"2024-05-11T01:01:45.295261+0000","last_unstale":"2024-05-11T01:01:45.300594+0000","last_undegraded":"2024-05-11T01:01:45.300594+0000","last_fullsized":"2024-05-11T01:01:45.300594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:14.953184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.278486+0000","last_change":"2024-05-11T01:01:44.282707+0000","last_active":"2024-05-11T01:01:45.278486+0000","last_peered":"2024-05-11T01:01:45.278486+0000","last_clean":"2024-05-11T01:01:45.278486+0000","last_became_active":"2024-05-11T01:01:44.282174+0000","last_became_peered":"2024-05-11T01:01:44.282174+0000","last_unstale":"2024-05-11T01:01:45.278486+0000","last_undegraded":"2024-05-11T01:01:45.278486+0000","last_fullsized":"2024-05-11T01:01:45.278486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:21:20.490794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299361+0000","last_change":"2024-05-11T01:01:45.295924+0000","last_active":"2024-05-11T01:01:45.299361+0000","last_peered":"2024-05-11T01:01:45.299361+0000","last_clean":"2024-05-11T01:01:45.299361+0000","last_became_active":"2024-05-11T01:01:45.295072+0000","last_became_peered":"2024-05-11T01:01:45.295072+0000","last_unstale":"2024-05-11T01:01:45.299361+0000","last_undegraded":"2024-05-11T01:01:45.299361+0000","last_fullsized":"2024-05-11T01:01:45.299361+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:43:01.685079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.276462+0000","last_change":"2024-05-11T01:01:44.288428+0000","last_active":"2024-05-11T01:01:45.276462+0000","last_peered":"2024-05-11T01:01:45.276462+0000","last_clean":"2024-05-11T01:01:45.276462+0000","last_became_active":"2024-05-11T01:01:44.287474+0000","last_became_peered":"2024-05-11T01:01:44.287474+0000","last_unstale":"2024-05-11T01:01:45.276462+0000","last_undegraded":"2024-05-11T01:01:45.276462+0000","last_fullsized":"2024-05-11T01:01:45.276462+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:39.042079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.298940+0000","last_change":"2024-05-11T01:01:45.296282+0000","last_active":"2024-05-11T01:01:45.298940+0000","last_peered":"2024-05-11T01:01:45.298940+0000","last_clean":"2024-05-11T01:01:45.298940+0000","last_became_active":"2024-05-11T01:01:45.296019+0000","last_became_peered":"2024-05-11T01:01:45.296019+0000","last_unstale":"2024-05-11T01:01:45.298940+0000","last_undegraded":"2024-05-11T01:01:45.298940+0000","last_fullsized":"2024-05-11T01:01:45.298940+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:23:44.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.307691+0000","last_change":"2024-05-11T01:01:44.284268+0000","last_active":"2024-05-11T01:01:46.307691+0000","last_peered":"2024-05-11T01:01:46.307691+0000","last_clean":"2024-05-11T01:01:46.307691+0000","last_became_active":"2024-05-11T01:01:44.283918+0000","last_became_peered":"2024-05-11T01:01:44.283918+0000","last_unstale":"2024-05-11T01:01:46.307691+0000","last_undegraded":"2024-05-11T01:01:46.307691+0000","last_fullsized":"2024-05-11T01:01:46.307691+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:17:19.132665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":127,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.282110+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:01:45.282110+0000","last_peered":"2024-05-11T01:01:45.282110+0000","last_clean":"2024-05-11T01:01:45.282110+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:01:45.282110+0000","last_undegraded":"2024-05-11T01:01:45.282110+0000","last_fullsized":"2024-05-11T01:01:45.282110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":128162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":711139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":128218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":127650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":129180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31500,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340340,"statfs":{"total":96636764160,"available":96604508160,"internally_reserved":0,"allocated":323584,"data_stored":120771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31500,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340340,"statfs":{"total":96636764160,"available":96604508160,"internally_reserved":0,"allocated":323584,"data_stored":120771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":711139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:01:49.337 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:01:49.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.476+0000 7f1ba986b640 1 Processor -- start 2024-05-11T01:01:49.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.477+0000 7f1ba986b640 1 -- start start 2024-05-11T01:01:49.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.477+0000 7f1ba986b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40d2ea0 0x7f1ba40d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.477+0000 7f1ba986b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba40d1f90 con 0x7f1ba415f2d0 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.477+0000 7f1ba986b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba40d2270 con 0x7f1ba40b2600 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.477+0000 7f1ba986b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba40d2550 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.477+0000 7f1ba2ffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f1ba415f2d0 0x7f1ba406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33876/0 (socket says 172.21.15.52:33876) 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.477+0000 7f1ba2ffd640 1 -- 172.21.15.52:0/3771963049 learned_addr learned my addr 172.21.15.52:0/3771963049 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.478+0000 7f1ba27fc640 1 --2- 172.21.15.52:0/3771963049 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40d2ea0 0x7f1ba40d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.478+0000 7f1ba1ffb640 1 -- 172.21.15.52:0/3771963049 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3245827248 0 0) 0x7f1ba40d1f90 con 0x7f1ba415f2d0 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.478+0000 7f1ba1ffb640 1 -- 172.21.15.52:0/3771963049 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b90003520 con 0x7f1ba415f2d0 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.478+0000 7f1ba1ffb640 1 -- 172.21.15.52:0/3771963049 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4104997493 0 0) 0x7f1ba40d2270 con 0x7f1ba40b2600 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.478+0000 7f1ba1ffb640 1 -- 172.21.15.52:0/3771963049 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ba40d1f90 con 0x7f1ba40b2600 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.478+0000 7f1ba27fc640 1 -- 172.21.15.52:0/3771963049 >> v1:172.21.15.119:6789/0 conn(0x7f1ba40b2600 legacy=0x7f1ba40b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.478+0000 7f1ba27fc640 1 -- 172.21.15.52:0/3771963049 >> v1:172.21.15.52:6789/0 conn(0x7f1ba415f2d0 legacy=0x7f1ba406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.478+0000 7f1ba1ffb640 1 -- 172.21.15.52:0/3771963049 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 806440364 0 0) 0x7f1b90003520 con 0x7f1ba415f2d0 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.478+0000 7f1ba27fc640 1 -- 172.21.15.52:0/3771963049 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba40d2830 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.479+0000 7f1ba27fc640 1 --2- 172.21.15.52:0/3771963049 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40d2ea0 0x7f1ba40d3330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0097d0 tx=0x7f1b8c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a859e8a32c0b61fe server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:49.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.479+0000 7f1ba1ffb640 1 -- 172.21.15.52:0/3771963049 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c037070 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.479+0000 7f1ba1ffb640 1 -- 172.21.15.52:0/3771963049 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c02ac30 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.479+0000 7f1ba1ffb640 1 -- 172.21.15.52:0/3771963049 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c001a70 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.479+0000 7f1ba986b640 1 -- 172.21.15.52:0/3771963049 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40d2ea0 msgr2=0x7f1ba40d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.479+0000 7f1ba986b640 1 --2- 172.21.15.52:0/3771963049 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40d2ea0 0x7f1ba40d3330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0097d0 tx=0x7f1b8c02a1a0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.479+0000 7f1ba986b640 1 -- 172.21.15.52:0/3771963049 shutdown_connections 2024-05-11T01:01:49.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.479+0000 7f1ba986b640 1 --2- 172.21.15.52:0/3771963049 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40d2ea0 0x7f1ba40d3330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.479+0000 7f1ba986b640 1 -- 172.21.15.52:0/3771963049 >> 172.21.15.52:0/3771963049 conn(0x7f1ba409ba20 msgr2=0x7f1ba409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:49.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.480+0000 7f1ba986b640 1 -- 172.21.15.52:0/3771963049 shutdown_connections 2024-05-11T01:01:49.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.480+0000 7f1ba986b640 1 -- 172.21.15.52:0/3771963049 wait complete. 2024-05-11T01:01:49.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.481+0000 7f1ba986b640 1 Processor -- start 2024-05-11T01:01:49.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.481+0000 7f1ba986b640 1 -- start start 2024-05-11T01:01:49.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba986b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40b2600 0x7f1ba40b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba986b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ba40d2ea0 0x7f1ba40716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba986b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ba415f2d0 0x7f1ba4071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba986b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba415f830 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba986b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba415f6b0 con 0x7f1ba415f2d0 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba986b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba40d1b80 con 0x7f1ba40b2600 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ba40d2ea0 0x7f1ba40716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ba40d2ea0 0x7f1ba40716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40854/0 (socket says 172.21.15.52:40854) 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba27fc640 1 -- 172.21.15.52:0/2955094425 learned_addr learned my addr 172.21.15.52:0/2955094425 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba2ffd640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40b2600 0x7f1ba40b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba37fe640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ba415f2d0 0x7f1ba4071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba27fc640 1 -- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40b2600 msgr2=0x7f1ba40b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba27fc640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40b2600 0x7f1ba40b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba27fc640 1 -- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ba415f2d0 msgr2=0x7f1ba4071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba27fc640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ba415f2d0 0x7f1ba4071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba27fc640 1 -- 172.21.15.52:0/2955094425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba40b76a0 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba37fe640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ba415f2d0 0x7f1ba4071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba27fc640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ba40d2ea0 0x7f1ba40716c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c035040 tx=0x7f1b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:49.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1b9bfff640 1 -- 172.21.15.52:0/2955094425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c037050 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1b9bfff640 1 -- 172.21.15.52:0/2955094425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c043070 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba40b5c90 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1b9bfff640 1 -- 172.21.15.52:0/2955094425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c006e10 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.482+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba40b6250 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.484+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba4069780 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.484+0000 7f1b9bfff640 1 -- 172.21.15.52:0/2955094425 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1b8c03ca30 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.485+0000 7f1b9bfff640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f1b840762d0 0x7f1b84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.485+0000 7f1b9bfff640 1 -- 172.21.15.52:0/2955094425 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f1b8c0b7fe0 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.485+0000 7f1ba2ffd640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f1b840762d0 0x7f1b84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.486+0000 7f1ba2ffd640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f1b840762d0 0x7f1b84078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0015a0 tx=0x7f1b9c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:49.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.489+0000 7f1b9bfff640 1 -- 172.21.15.52:0/2955094425 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1ba40b76a0 con 0x7f1ba40d2ea0 2024-05-11T01:01:49.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.679+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ba406e160 con 0x7f1b840762d0 2024-05-11T01:01:49.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.685+0000 7f1b9bfff640 1 -- 172.21.15.52:0/2955094425 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7f1ba406e160 con 0x7f1b840762d0 2024-05-11T01:01:49.688 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:49.689 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:01:49.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f1b840762d0 msgr2=0x7f1b84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f1b840762d0 0x7f1b84078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0015a0 tx=0x7f1b9c009040 comp rx=0 tx=0).stop 2024-05-11T01:01:49.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ba40d2ea0 msgr2=0x7f1ba40716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ba40d2ea0 0x7f1ba40716c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c035040 tx=0x7f1b8c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 shutdown_connections 2024-05-11T01:01:49.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f1b840762d0 0x7f1b84078790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ba415f2d0 0x7f1ba4071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ba40d2ea0 0x7f1ba40716c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 --2- 172.21.15.52:0/2955094425 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ba40b2600 0x7f1ba40b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 >> 172.21.15.52:0/2955094425 conn(0x7f1ba409ba20 msgr2=0x7f1ba415e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:49.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 shutdown_connections 2024-05-11T01:01:49.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.689+0000 7f1ba986b640 1 -- 172.21.15.52:0/2955094425 wait complete. 2024-05-11T01:01:49.700 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-05-11T01:01:49.520784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236584,"kb_used_data":3776,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738136,"statfs":{"total":773094113280,"available":772851851264,"internally_reserved":0,"allocated":3866624,"data_stored":2177030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001071"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308416+0000","last_change":"2024-05-11T01:01:45.292273+0000","last_active":"2024-05-11T01:01:46.308416+0000","last_peered":"2024-05-11T01:01:46.308416+0000","last_clean":"2024-05-11T01:01:46.308416+0000","last_became_active":"2024-05-11T01:01:45.292019+0000","last_became_peered":"2024-05-11T01:01:45.292019+0000","last_unstale":"2024-05-11T01:01:46.308416+0000","last_undegraded":"2024-05-11T01:01:46.308416+0000","last_fullsized":"2024-05-11T01:01:46.308416+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:24.197518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.295181+0000","last_change":"2024-05-11T01:01:45.289886+0000","last_active":"2024-05-11T01:01:45.295181+0000","last_peered":"2024-05-11T01:01:45.295181+0000","last_clean":"2024-05-11T01:01:45.295181+0000","last_became_active":"2024-05-11T01:01:45.289413+0000","last_became_peered":"2024-05-11T01:01:45.289413+0000","last_unstale":"2024-05-11T01:01:45.295181+0000","last_undegraded":"2024-05-11T01:01:45.295181+0000","last_fullsized":"2024-05-11T01:01:45.295181+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:11:53.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289939+0000","last_change":"2024-05-11T01:01:45.286186+0000","last_active":"2024-05-11T01:01:45.289939+0000","last_peered":"2024-05-11T01:01:45.289939+0000","last_clean":"2024-05-11T01:01:45.289939+0000","last_became_active":"2024-05-11T01:01:45.285897+0000","last_became_peered":"2024-05-11T01:01:45.285897+0000","last_unstale":"2024-05-11T01:01:45.289939+0000","last_undegraded":"2024-05-11T01:01:45.289939+0000","last_fullsized":"2024-05-11T01:01:45.289939+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:23:44.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.296135+0000","last_change":"2024-05-11T01:01:45.292553+0000","last_active":"2024-05-11T01:01:45.296135+0000","last_peered":"2024-05-11T01:01:45.296135+0000","last_clean":"2024-05-11T01:01:45.296135+0000","last_became_active":"2024-05-11T01:01:45.292224+0000","last_became_peered":"2024-05-11T01:01:45.292224+0000","last_unstale":"2024-05-11T01:01:45.296135+0000","last_undegraded":"2024-05-11T01:01:45.296135+0000","last_fullsized":"2024-05-11T01:01:45.296135+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:24.197518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301408+0000","last_change":"2024-05-11T01:01:45.298505+0000","last_active":"2024-05-11T01:01:45.301408+0000","last_peered":"2024-05-11T01:01:45.301408+0000","last_clean":"2024-05-11T01:01:45.301408+0000","last_became_active":"2024-05-11T01:01:45.298012+0000","last_became_peered":"2024-05-11T01:01:45.298012+0000","last_unstale":"2024-05-11T01:01:45.301408+0000","last_undegraded":"2024-05-11T01:01:45.301408+0000","last_fullsized":"2024-05-11T01:01:45.301408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:56.599542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305466+0000","last_change":"2024-05-11T01:01:45.299253+0000","last_active":"2024-05-11T01:01:45.305466+0000","last_peered":"2024-05-11T01:01:45.305466+0000","last_clean":"2024-05-11T01:01:45.305466+0000","last_became_active":"2024-05-11T01:01:45.298975+0000","last_became_peered":"2024-05-11T01:01:45.298975+0000","last_unstale":"2024-05-11T01:01:45.305466+0000","last_undegraded":"2024-05-11T01:01:45.305466+0000","last_fullsized":"2024-05-11T01:01:45.305466+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:17:32.179452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.298813+0000","last_change":"2024-05-11T01:01:45.295748+0000","last_active":"2024-05-11T01:01:45.298813+0000","last_peered":"2024-05-11T01:01:45.298813+0000","last_clean":"2024-05-11T01:01:45.298813+0000","last_became_active":"2024-05-11T01:01:45.293477+0000","last_became_peered":"2024-05-11T01:01:45.293477+0000","last_unstale":"2024-05-11T01:01:45.298813+0000","last_undegraded":"2024-05-11T01:01:45.298813+0000","last_fullsized":"2024-05-11T01:01:45.298813+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:12:42.551695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.292147+0000","last_change":"2024-05-11T01:01:45.290112+0000","last_active":"2024-05-11T01:01:45.292147+0000","last_peered":"2024-05-11T01:01:45.292147+0000","last_clean":"2024-05-11T01:01:45.292147+0000","last_became_active":"2024-05-11T01:01:45.289808+0000","last_became_peered":"2024-05-11T01:01:45.289808+0000","last_unstale":"2024-05-11T01:01:45.292147+0000","last_undegraded":"2024-05-11T01:01:45.292147+0000","last_fullsized":"2024-05-11T01:01:45.292147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:24.197518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.288384+0000","last_change":"2024-05-11T01:01:45.285375+0000","last_active":"2024-05-11T01:01:45.288384+0000","last_peered":"2024-05-11T01:01:45.288384+0000","last_clean":"2024-05-11T01:01:45.288384+0000","last_became_active":"2024-05-11T01:01:45.284943+0000","last_became_peered":"2024-05-11T01:01:45.284943+0000","last_unstale":"2024-05-11T01:01:45.288384+0000","last_undegraded":"2024-05-11T01:01:45.288384+0000","last_fullsized":"2024-05-11T01:01:45.288384+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:02:37.642922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.304077+0000","last_change":"2024-05-11T01:01:45.300881+0000","last_active":"2024-05-11T01:01:45.304077+0000","last_peered":"2024-05-11T01:01:45.304077+0000","last_clean":"2024-05-11T01:01:45.304077+0000","last_became_active":"2024-05-11T01:01:45.299898+0000","last_became_peered":"2024-05-11T01:01:45.299898+0000","last_unstale":"2024-05-11T01:01:45.304077+0000","last_undegraded":"2024-05-11T01:01:45.304077+0000","last_fullsized":"2024-05-11T01:01:45.304077+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:12:42.551695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289239+0000","last_change":"2024-05-11T01:01:45.288279+0000","last_active":"2024-05-11T01:01:45.289239+0000","last_peered":"2024-05-11T01:01:45.289239+0000","last_clean":"2024-05-11T01:01:45.289239+0000","last_became_active":"2024-05-11T01:01:45.287969+0000","last_became_peered":"2024-05-11T01:01:45.287969+0000","last_unstale":"2024-05-11T01:01:45.289239+0000","last_undegraded":"2024-05-11T01:01:45.289239+0000","last_fullsized":"2024-05-11T01:01:45.289239+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:40.047660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.292803+0000","last_change":"2024-05-11T01:01:45.286582+0000","last_active":"2024-05-11T01:01:45.292803+0000","last_peered":"2024-05-11T01:01:45.292803+0000","last_clean":"2024-05-11T01:01:45.292803+0000","last_became_active":"2024-05-11T01:01:45.286338+0000","last_became_peered":"2024-05-11T01:01:45.286338+0000","last_unstale":"2024-05-11T01:01:45.292803+0000","last_undegraded":"2024-05-11T01:01:45.292803+0000","last_fullsized":"2024-05-11T01:01:45.292803+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:17.182079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301390+0000","last_change":"2024-05-11T01:01:45.290183+0000","last_active":"2024-05-11T01:01:45.301390+0000","last_peered":"2024-05-11T01:01:45.301390+0000","last_clean":"2024-05-11T01:01:45.301390+0000","last_became_active":"2024-05-11T01:01:45.288298+0000","last_became_peered":"2024-05-11T01:01:45.288298+0000","last_unstale":"2024-05-11T01:01:45.301390+0000","last_undegraded":"2024-05-11T01:01:45.301390+0000","last_fullsized":"2024-05-11T01:01:45.301390+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:14:32.666306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305816+0000","last_change":"2024-05-11T01:01:45.304408+0000","last_active":"2024-05-11T01:01:45.305816+0000","last_peered":"2024-05-11T01:01:45.305816+0000","last_clean":"2024-05-11T01:01:45.305816+0000","last_became_active":"2024-05-11T01:01:45.304125+0000","last_became_peered":"2024-05-11T01:01:45.304125+0000","last_unstale":"2024-05-11T01:01:45.305816+0000","last_undegraded":"2024-05-11T01:01:45.305816+0000","last_fullsized":"2024-05-11T01:01:45.305816+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:33:40.871894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.292295+0000","last_change":"2024-05-11T01:01:45.287732+0000","last_active":"2024-05-11T01:01:45.292295+0000","last_peered":"2024-05-11T01:01:45.292295+0000","last_clean":"2024-05-11T01:01:45.292295+0000","last_became_active":"2024-05-11T01:01:45.287438+0000","last_became_peered":"2024-05-11T01:01:45.287438+0000","last_unstale":"2024-05-11T01:01:45.292295+0000","last_undegraded":"2024-05-11T01:01:45.292295+0000","last_fullsized":"2024-05-11T01:01:45.292295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:14.953184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.306374+0000","last_change":"2024-05-11T01:01:45.300595+0000","last_active":"2024-05-11T01:01:45.306374+0000","last_peered":"2024-05-11T01:01:45.306374+0000","last_clean":"2024-05-11T01:01:45.306374+0000","last_became_active":"2024-05-11T01:01:45.300361+0000","last_became_peered":"2024-05-11T01:01:45.300361+0000","last_unstale":"2024-05-11T01:01:45.306374+0000","last_undegraded":"2024-05-11T01:01:45.306374+0000","last_fullsized":"2024-05-11T01:01:45.306374+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:07:10.746755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.294100+0000","last_change":"2024-05-11T01:01:45.290986+0000","last_active":"2024-05-11T01:01:45.294100+0000","last_peered":"2024-05-11T01:01:45.294100+0000","last_clean":"2024-05-11T01:01:45.294100+0000","last_became_active":"2024-05-11T01:01:45.290579+0000","last_became_peered":"2024-05-11T01:01:45.290579+0000","last_unstale":"2024-05-11T01:01:45.294100+0000","last_undegraded":"2024-05-11T01:01:45.294100+0000","last_fullsized":"2024-05-11T01:01:45.294100+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.295161+0000","last_change":"2024-05-11T01:01:45.284671+0000","last_active":"2024-05-11T01:01:45.295161+0000","last_peered":"2024-05-11T01:01:45.295161+0000","last_clean":"2024-05-11T01:01:45.295161+0000","last_became_active":"2024-05-11T01:01:45.283784+0000","last_became_peered":"2024-05-11T01:01:45.283784+0000","last_unstale":"2024-05-11T01:01:45.295161+0000","last_undegraded":"2024-05-11T01:01:45.295161+0000","last_fullsized":"2024-05-11T01:01:45.295161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.300516+0000","last_change":"2024-05-11T01:01:45.296460+0000","last_active":"2024-05-11T01:01:45.300516+0000","last_peered":"2024-05-11T01:01:45.300516+0000","last_clean":"2024-05-11T01:01:45.300516+0000","last_became_active":"2024-05-11T01:01:45.296154+0000","last_became_peered":"2024-05-11T01:01:45.296154+0000","last_unstale":"2024-05-11T01:01:45.300516+0000","last_undegraded":"2024-05-11T01:01:45.300516+0000","last_fullsized":"2024-05-11T01:01:45.300516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:49:08.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301303+0000","last_change":"2024-05-11T01:01:45.293455+0000","last_active":"2024-05-11T01:01:45.301303+0000","last_peered":"2024-05-11T01:01:45.301303+0000","last_clean":"2024-05-11T01:01:45.301303+0000","last_became_active":"2024-05-11T01:01:45.293133+0000","last_became_peered":"2024-05-11T01:01:45.293133+0000","last_unstale":"2024-05-11T01:01:45.301303+0000","last_undegraded":"2024-05-11T01:01:45.301303+0000","last_fullsized":"2024-05-11T01:01:45.301303+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:53.959890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.297957+0000","last_change":"2024-05-11T01:01:45.290392+0000","last_active":"2024-05-11T01:01:45.297957+0000","last_peered":"2024-05-11T01:01:45.297957+0000","last_clean":"2024-05-11T01:01:45.297957+0000","last_became_active":"2024-05-11T01:01:45.290160+0000","last_became_peered":"2024-05-11T01:01:45.290160+0000","last_unstale":"2024-05-11T01:01:45.297957+0000","last_undegraded":"2024-05-11T01:01:45.297957+0000","last_fullsized":"2024-05-11T01:01:45.297957+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:24.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299305+0000","last_change":"2024-05-11T01:01:45.297443+0000","last_active":"2024-05-11T01:01:45.299305+0000","last_peered":"2024-05-11T01:01:45.299305+0000","last_clean":"2024-05-11T01:01:45.299305+0000","last_became_active":"2024-05-11T01:01:45.296817+0000","last_became_peered":"2024-05-11T01:01:45.296817+0000","last_unstale":"2024-05-11T01:01:45.299305+0000","last_undegraded":"2024-05-11T01:01:45.299305+0000","last_fullsized":"2024-05-11T01:01:45.299305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:58.923752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.303498+0000","last_change":"2024-05-11T01:01:45.302563+0000","last_active":"2024-05-11T01:01:45.303498+0000","last_peered":"2024-05-11T01:01:45.303498+0000","last_clean":"2024-05-11T01:01:45.303498+0000","last_became_active":"2024-05-11T01:01:45.302249+0000","last_became_peered":"2024-05-11T01:01:45.302249+0000","last_unstale":"2024-05-11T01:01:45.303498+0000","last_undegraded":"2024-05-11T01:01:45.303498+0000","last_fullsized":"2024-05-11T01:01:45.303498+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:17:11.351660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289737+0000","last_change":"2024-05-11T01:01:45.282806+0000","last_active":"2024-05-11T01:01:45.289737+0000","last_peered":"2024-05-11T01:01:45.289737+0000","last_clean":"2024-05-11T01:01:45.289737+0000","last_became_active":"2024-05-11T01:01:45.282583+0000","last_became_peered":"2024-05-11T01:01:45.282583+0000","last_unstale":"2024-05-11T01:01:45.289737+0000","last_undegraded":"2024-05-11T01:01:45.289737+0000","last_fullsized":"2024-05-11T01:01:45.289737+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.296508+0000","last_change":"2024-05-11T01:01:45.291086+0000","last_active":"2024-05-11T01:01:45.296508+0000","last_peered":"2024-05-11T01:01:45.296508+0000","last_clean":"2024-05-11T01:01:45.296508+0000","last_became_active":"2024-05-11T01:01:45.290335+0000","last_became_peered":"2024-05-11T01:01:45.290335+0000","last_unstale":"2024-05-11T01:01:45.296508+0000","last_undegraded":"2024-05-11T01:01:45.296508+0000","last_fullsized":"2024-05-11T01:01:45.296508+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:33.401596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.304678+0000","last_change":"2024-05-11T01:01:45.296462+0000","last_active":"2024-05-11T01:01:45.304678+0000","last_peered":"2024-05-11T01:01:45.304678+0000","last_clean":"2024-05-11T01:01:45.304678+0000","last_became_active":"2024-05-11T01:01:45.294415+0000","last_became_peered":"2024-05-11T01:01:45.294415+0000","last_unstale":"2024-05-11T01:01:45.304678+0000","last_undegraded":"2024-05-11T01:01:45.304678+0000","last_fullsized":"2024-05-11T01:01:45.304678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:07:10.746755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.286402+0000","last_change":"2024-05-11T01:01:45.282837+0000","last_active":"2024-05-11T01:01:45.286402+0000","last_peered":"2024-05-11T01:01:45.286402+0000","last_clean":"2024-05-11T01:01:45.286402+0000","last_became_active":"2024-05-11T01:01:45.281896+0000","last_became_peered":"2024-05-11T01:01:45.281896+0000","last_unstale":"2024-05-11T01:01:45.286402+0000","last_undegraded":"2024-05-11T01:01:45.286402+0000","last_fullsized":"2024-05-11T01:01:45.286402+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:58.923752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299271+0000","last_change":"2024-05-11T01:01:45.292996+0000","last_active":"2024-05-11T01:01:45.299271+0000","last_peered":"2024-05-11T01:01:45.299271+0000","last_clean":"2024-05-11T01:01:45.299271+0000","last_became_active":"2024-05-11T01:01:45.292623+0000","last_became_peered":"2024-05-11T01:01:45.292623+0000","last_unstale":"2024-05-11T01:01:45.299271+0000","last_undegraded":"2024-05-11T01:01:45.299271+0000","last_fullsized":"2024-05-11T01:01:45.299271+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:49:08.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299050+0000","last_change":"2024-05-11T01:01:45.297337+0000","last_active":"2024-05-11T01:01:45.299050+0000","last_peered":"2024-05-11T01:01:45.299050+0000","last_clean":"2024-05-11T01:01:45.299050+0000","last_became_active":"2024-05-11T01:01:45.297031+0000","last_became_peered":"2024-05-11T01:01:45.297031+0000","last_unstale":"2024-05-11T01:01:45.299050+0000","last_undegraded":"2024-05-11T01:01:45.299050+0000","last_fullsized":"2024-05-11T01:01:45.299050+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:06.518581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289215+0000","last_change":"2024-05-11T01:01:45.288295+0000","last_active":"2024-05-11T01:01:45.289215+0000","last_peered":"2024-05-11T01:01:45.289215+0000","last_clean":"2024-05-11T01:01:45.289215+0000","last_became_active":"2024-05-11T01:01:45.288000+0000","last_became_peered":"2024-05-11T01:01:45.288000+0000","last_unstale":"2024-05-11T01:01:45.289215+0000","last_undegraded":"2024-05-11T01:01:45.289215+0000","last_fullsized":"2024-05-11T01:01:45.289215+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:33.401596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308034+0000","last_change":"2024-05-11T01:01:45.290840+0000","last_active":"2024-05-11T01:01:46.308034+0000","last_peered":"2024-05-11T01:01:46.308034+0000","last_clean":"2024-05-11T01:01:46.308034+0000","last_became_active":"2024-05-11T01:01:45.290214+0000","last_became_peered":"2024-05-11T01:01:45.290214+0000","last_unstale":"2024-05-11T01:01:46.308034+0000","last_undegraded":"2024-05-11T01:01:46.308034+0000","last_fullsized":"2024-05-11T01:01:46.308034+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:58.923752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299310+0000","last_change":"2024-05-11T01:01:45.297105+0000","last_active":"2024-05-11T01:01:45.299310+0000","last_peered":"2024-05-11T01:01:45.299310+0000","last_clean":"2024-05-11T01:01:45.299310+0000","last_became_active":"2024-05-11T01:01:45.296786+0000","last_became_peered":"2024-05-11T01:01:45.296786+0000","last_unstale":"2024-05-11T01:01:45.299310+0000","last_undegraded":"2024-05-11T01:01:45.299310+0000","last_fullsized":"2024-05-11T01:01:45.299310+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:17.182079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.302763+0000","last_change":"2024-05-11T01:01:45.301925+0000","last_active":"2024-05-11T01:01:45.302763+0000","last_peered":"2024-05-11T01:01:45.302763+0000","last_clean":"2024-05-11T01:01:45.302763+0000","last_became_active":"2024-05-11T01:01:45.301643+0000","last_became_peered":"2024-05-11T01:01:45.301643+0000","last_unstale":"2024-05-11T01:01:45.302763+0000","last_undegraded":"2024-05-11T01:01:45.302763+0000","last_fullsized":"2024-05-11T01:01:45.302763+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:02:37.642922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308657+0000","last_change":"2024-05-11T01:01:44.285791+0000","last_active":"2024-05-11T01:01:46.308657+0000","last_peered":"2024-05-11T01:01:46.308657+0000","last_clean":"2024-05-11T01:01:46.308657+0000","last_became_active":"2024-05-11T01:01:44.285482+0000","last_became_peered":"2024-05-11T01:01:44.285482+0000","last_unstale":"2024-05-11T01:01:46.308657+0000","last_undegraded":"2024-05-11T01:01:46.308657+0000","last_fullsized":"2024-05-11T01:01:46.308657+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:50.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308555+0000","last_change":"2024-05-11T01:01:45.295694+0000","last_active":"2024-05-11T01:01:46.308555+0000","last_peered":"2024-05-11T01:01:46.308555+0000","last_clean":"2024-05-11T01:01:46.308555+0000","last_became_active":"2024-05-11T01:01:45.295432+0000","last_became_peered":"2024-05-11T01:01:45.295432+0000","last_unstale":"2024-05-11T01:01:46.308555+0000","last_undegraded":"2024-05-11T01:01:46.308555+0000","last_fullsized":"2024-05-11T01:01:46.308555+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.307288+0000","last_change":"2024-05-11T01:01:44.286162+0000","last_active":"2024-05-11T01:01:45.307288+0000","last_peered":"2024-05-11T01:01:45.307288+0000","last_clean":"2024-05-11T01:01:45.307288+0000","last_became_active":"2024-05-11T01:01:44.285818+0000","last_became_peered":"2024-05-11T01:01:44.285818+0000","last_unstale":"2024-05-11T01:01:45.307288+0000","last_undegraded":"2024-05-11T01:01:45.307288+0000","last_fullsized":"2024-05-11T01:01:45.307288+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:21:20.490794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305604+0000","last_change":"2024-05-11T01:01:45.304646+0000","last_active":"2024-05-11T01:01:45.305604+0000","last_peered":"2024-05-11T01:01:45.305604+0000","last_clean":"2024-05-11T01:01:45.305604+0000","last_became_active":"2024-05-11T01:01:45.304404+0000","last_became_peered":"2024-05-11T01:01:45.304404+0000","last_unstale":"2024-05-11T01:01:45.305604+0000","last_undegraded":"2024-05-11T01:01:45.305604+0000","last_fullsized":"2024-05-11T01:01:45.305604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:40.047660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.304029+0000","last_change":"2024-05-11T01:01:44.290314+0000","last_active":"2024-05-11T01:01:45.304029+0000","last_peered":"2024-05-11T01:01:45.304029+0000","last_clean":"2024-05-11T01:01:45.304029+0000","last_became_active":"2024-05-11T01:01:44.290042+0000","last_became_peered":"2024-05-11T01:01:44.290042+0000","last_unstale":"2024-05-11T01:01:45.304029+0000","last_undegraded":"2024-05-11T01:01:45.304029+0000","last_fullsized":"2024-05-11T01:01:45.304029+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:05.512999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.286425+0000","last_change":"2024-05-11T01:01:45.282871+0000","last_active":"2024-05-11T01:01:45.286425+0000","last_peered":"2024-05-11T01:01:45.286425+0000","last_clean":"2024-05-11T01:01:45.286425+0000","last_became_active":"2024-05-11T01:01:45.281469+0000","last_became_peered":"2024-05-11T01:01:45.281469+0000","last_unstale":"2024-05-11T01:01:45.286425+0000","last_undegraded":"2024-05-11T01:01:45.286425+0000","last_fullsized":"2024-05-11T01:01:45.286425+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:40.047660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.307337+0000","last_change":"2024-05-11T01:01:44.291756+0000","last_active":"2024-05-11T01:01:45.307337+0000","last_peered":"2024-05-11T01:01:45.307337+0000","last_clean":"2024-05-11T01:01:45.307337+0000","last_became_active":"2024-05-11T01:01:44.291428+0000","last_became_peered":"2024-05-11T01:01:44.291428+0000","last_unstale":"2024-05-11T01:01:45.307337+0000","last_undegraded":"2024-05-11T01:01:45.307337+0000","last_fullsized":"2024-05-11T01:01:45.307337+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:37:15.659807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.306324+0000","last_change":"2024-05-11T01:01:45.298629+0000","last_active":"2024-05-11T01:01:45.306324+0000","last_peered":"2024-05-11T01:01:45.306324+0000","last_clean":"2024-05-11T01:01:45.306324+0000","last_became_active":"2024-05-11T01:01:45.298312+0000","last_became_peered":"2024-05-11T01:01:45.298312+0000","last_unstale":"2024-05-11T01:01:45.306324+0000","last_undegraded":"2024-05-11T01:01:45.306324+0000","last_fullsized":"2024-05-11T01:01:45.306324+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:14.953184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305837+0000","last_change":"2024-05-11T01:01:44.288609+0000","last_active":"2024-05-11T01:01:45.305837+0000","last_peered":"2024-05-11T01:01:45.305837+0000","last_clean":"2024-05-11T01:01:45.305837+0000","last_became_active":"2024-05-11T01:01:44.288308+0000","last_became_peered":"2024-05-11T01:01:44.288308+0000","last_unstale":"2024-05-11T01:01:45.305837+0000","last_undegraded":"2024-05-11T01:01:45.305837+0000","last_fullsized":"2024-05-11T01:01:45.305837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:17:19.132665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299021+0000","last_change":"2024-05-11T01:01:45.293175+0000","last_active":"2024-05-11T01:01:45.299021+0000","last_peered":"2024-05-11T01:01:45.299021+0000","last_clean":"2024-05-11T01:01:45.299021+0000","last_became_active":"2024-05-11T01:01:45.292801+0000","last_became_peered":"2024-05-11T01:01:45.292801+0000","last_unstale":"2024-05-11T01:01:45.299021+0000","last_undegraded":"2024-05-11T01:01:45.299021+0000","last_fullsized":"2024-05-11T01:01:45.299021+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:07:10.746755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305828+0000","last_change":"2024-05-11T01:01:44.287857+0000","last_active":"2024-05-11T01:01:45.305828+0000","last_peered":"2024-05-11T01:01:45.305828+0000","last_clean":"2024-05-11T01:01:45.305828+0000","last_became_active":"2024-05-11T01:01:44.287580+0000","last_became_peered":"2024-05-11T01:01:44.287580+0000","last_unstale":"2024-05-11T01:01:45.305828+0000","last_undegraded":"2024-05-11T01:01:45.305828+0000","last_fullsized":"2024-05-11T01:01:45.305828+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:00:46.594424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.298037+0000","last_change":"2024-05-11T01:01:45.289642+0000","last_active":"2024-05-11T01:01:45.298037+0000","last_peered":"2024-05-11T01:01:45.298037+0000","last_clean":"2024-05-11T01:01:45.298037+0000","last_became_active":"2024-05-11T01:01:45.289324+0000","last_became_peered":"2024-05-11T01:01:45.289324+0000","last_unstale":"2024-05-11T01:01:45.298037+0000","last_undegraded":"2024-05-11T01:01:45.298037+0000","last_fullsized":"2024-05-11T01:01:45.298037+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:17:11.351660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.275039+0000","last_change":"2024-05-11T01:01:44.280527+0000","last_active":"2024-05-11T01:01:45.275039+0000","last_peered":"2024-05-11T01:01:45.275039+0000","last_clean":"2024-05-11T01:01:45.275039+0000","last_became_active":"2024-05-11T01:01:44.280236+0000","last_became_peered":"2024-05-11T01:01:44.280236+0000","last_unstale":"2024-05-11T01:01:45.275039+0000","last_undegraded":"2024-05-11T01:01:45.275039+0000","last_fullsized":"2024-05-11T01:01:45.275039+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:43:00.679498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.297355+0000","last_change":"2024-05-11T01:01:45.293282+0000","last_active":"2024-05-11T01:01:45.297355+0000","last_peered":"2024-05-11T01:01:45.297355+0000","last_clean":"2024-05-11T01:01:45.297355+0000","last_became_active":"2024-05-11T01:01:45.292766+0000","last_became_peered":"2024-05-11T01:01:45.292766+0000","last_unstale":"2024-05-11T01:01:45.297355+0000","last_undegraded":"2024-05-11T01:01:45.297355+0000","last_fullsized":"2024-05-11T01:01:45.297355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:11:53.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.278251+0000","last_change":"2024-05-11T01:01:44.285917+0000","last_active":"2024-05-11T01:01:45.278251+0000","last_peered":"2024-05-11T01:01:45.278251+0000","last_clean":"2024-05-11T01:01:45.278251+0000","last_became_active":"2024-05-11T01:01:44.285631+0000","last_became_peered":"2024-05-11T01:01:44.285631+0000","last_unstale":"2024-05-11T01:01:45.278251+0000","last_undegraded":"2024-05-11T01:01:45.278251+0000","last_fullsized":"2024-05-11T01:01:45.278251+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:23:52.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289468+0000","last_change":"2024-05-11T01:01:45.282249+0000","last_active":"2024-05-11T01:01:45.289468+0000","last_peered":"2024-05-11T01:01:45.289468+0000","last_clean":"2024-05-11T01:01:45.289468+0000","last_became_active":"2024-05-11T01:01:45.281956+0000","last_became_peered":"2024-05-11T01:01:45.281956+0000","last_unstale":"2024-05-11T01:01:45.289468+0000","last_undegraded":"2024-05-11T01:01:45.289468+0000","last_fullsized":"2024-05-11T01:01:45.289468+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:24.197518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.276109+0000","last_change":"2024-05-11T01:01:44.285505+0000","last_active":"2024-05-11T01:01:45.276109+0000","last_peered":"2024-05-11T01:01:45.276109+0000","last_clean":"2024-05-11T01:01:45.276109+0000","last_became_active":"2024-05-11T01:01:44.285240+0000","last_became_peered":"2024-05-11T01:01:44.285240+0000","last_unstale":"2024-05-11T01:01:45.276109+0000","last_undegraded":"2024-05-11T01:01:45.276109+0000","last_fullsized":"2024-05-11T01:01:45.276109+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:57.918170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.300483+0000","last_change":"2024-05-11T01:01:45.295564+0000","last_active":"2024-05-11T01:01:45.300483+0000","last_peered":"2024-05-11T01:01:45.300483+0000","last_clean":"2024-05-11T01:01:45.300483+0000","last_became_active":"2024-05-11T01:01:45.295237+0000","last_became_peered":"2024-05-11T01:01:45.295237+0000","last_unstale":"2024-05-11T01:01:45.300483+0000","last_undegraded":"2024-05-11T01:01:45.300483+0000","last_fullsized":"2024-05-11T01:01:45.300483+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:32:17.182079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.283602+0000","last_change":"2024-05-11T01:01:44.280751+0000","last_active":"2024-05-11T01:01:45.283602+0000","last_peered":"2024-05-11T01:01:45.283602+0000","last_clean":"2024-05-11T01:01:45.283602+0000","last_became_active":"2024-05-11T01:01:44.280479+0000","last_became_peered":"2024-05-11T01:01:44.280479+0000","last_unstale":"2024-05-11T01:01:45.283602+0000","last_undegraded":"2024-05-11T01:01:45.283602+0000","last_fullsized":"2024-05-11T01:01:45.283602+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:45:35.410817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.297604+0000","last_change":"2024-05-11T01:01:45.294751+0000","last_active":"2024-05-11T01:01:45.297604+0000","last_peered":"2024-05-11T01:01:45.297604+0000","last_clean":"2024-05-11T01:01:45.297604+0000","last_became_active":"2024-05-11T01:01:45.294429+0000","last_became_peered":"2024-05-11T01:01:45.294429+0000","last_unstale":"2024-05-11T01:01:45.297604+0000","last_undegraded":"2024-05-11T01:01:45.297604+0000","last_fullsized":"2024-05-11T01:01:45.297604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:23:44.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.296428+0000","last_change":"2024-05-11T01:01:44.287411+0000","last_active":"2024-05-11T01:01:45.296428+0000","last_peered":"2024-05-11T01:01:45.296428+0000","last_clean":"2024-05-11T01:01:45.296428+0000","last_became_active":"2024-05-11T01:01:44.286927+0000","last_became_peered":"2024-05-11T01:01:44.286927+0000","last_unstale":"2024-05-11T01:01:45.296428+0000","last_undegraded":"2024-05-11T01:01:45.296428+0000","last_fullsized":"2024-05-11T01:01:45.296428+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:17:57.918170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289919+0000","last_change":"2024-05-11T01:01:45.286021+0000","last_active":"2024-05-11T01:01:45.289919+0000","last_peered":"2024-05-11T01:01:45.289919+0000","last_clean":"2024-05-11T01:01:45.289919+0000","last_became_active":"2024-05-11T01:01:45.285721+0000","last_became_peered":"2024-05-11T01:01:45.285721+0000","last_unstale":"2024-05-11T01:01:45.289919+0000","last_undegraded":"2024-05-11T01:01:45.289919+0000","last_fullsized":"2024-05-11T01:01:45.289919+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:12:42.551695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.308523+0000","last_change":"2024-05-11T01:01:44.289402+0000","last_active":"2024-05-11T01:01:45.308523+0000","last_peered":"2024-05-11T01:01:45.308523+0000","last_clean":"2024-05-11T01:01:45.308523+0000","last_became_active":"2024-05-11T01:01:44.289115+0000","last_became_peered":"2024-05-11T01:01:44.289115+0000","last_unstale":"2024-05-11T01:01:45.308523+0000","last_undegraded":"2024-05-11T01:01:45.308523+0000","last_fullsized":"2024-05-11T01:01:45.308523+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:32.396014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.297394+0000","last_change":"2024-05-11T01:01:45.293221+0000","last_active":"2024-05-11T01:01:45.297394+0000","last_peered":"2024-05-11T01:01:45.297394+0000","last_clean":"2024-05-11T01:01:45.297394+0000","last_became_active":"2024-05-11T01:01:45.292910+0000","last_became_peered":"2024-05-11T01:01:45.292910+0000","last_unstale":"2024-05-11T01:01:45.297394+0000","last_undegraded":"2024-05-11T01:01:45.297394+0000","last_fullsized":"2024-05-11T01:01:45.297394+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:06:14.036543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301103+0000","last_change":"2024-05-11T01:01:44.287627+0000","last_active":"2024-05-11T01:01:45.301103+0000","last_peered":"2024-05-11T01:01:45.301103+0000","last_clean":"2024-05-11T01:01:45.301103+0000","last_became_active":"2024-05-11T01:01:44.287334+0000","last_became_peered":"2024-05-11T01:01:44.287334+0000","last_unstale":"2024-05-11T01:01:45.301103+0000","last_undegraded":"2024-05-11T01:01:45.301103+0000","last_fullsized":"2024-05-11T01:01:45.301103+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:45:35.410817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.294838+0000","last_change":"2024-05-11T01:01:45.290907+0000","last_active":"2024-05-11T01:01:45.294838+0000","last_peered":"2024-05-11T01:01:45.294838+0000","last_clean":"2024-05-11T01:01:45.294838+0000","last_became_active":"2024-05-11T01:01:45.290606+0000","last_became_peered":"2024-05-11T01:01:45.290606+0000","last_unstale":"2024-05-11T01:01:45.294838+0000","last_undegraded":"2024-05-11T01:01:45.294838+0000","last_fullsized":"2024-05-11T01:01:45.294838+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:12:42.551695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.277510+0000","last_change":"2024-05-11T01:01:44.289402+0000","last_active":"2024-05-11T01:01:45.277510+0000","last_peered":"2024-05-11T01:01:45.277510+0000","last_clean":"2024-05-11T01:01:45.277510+0000","last_became_active":"2024-05-11T01:01:44.289138+0000","last_became_peered":"2024-05-11T01:01:44.289138+0000","last_unstale":"2024-05-11T01:01:45.277510+0000","last_undegraded":"2024-05-11T01:01:45.277510+0000","last_fullsized":"2024-05-11T01:01:45.277510+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:17:10.346079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.306352+0000","last_change":"2024-05-11T01:01:45.299228+0000","last_active":"2024-05-11T01:01:45.306352+0000","last_peered":"2024-05-11T01:01:45.306352+0000","last_clean":"2024-05-11T01:01:45.306352+0000","last_became_active":"2024-05-11T01:01:45.299013+0000","last_became_peered":"2024-05-11T01:01:45.299013+0000","last_unstale":"2024-05-11T01:01:45.306352+0000","last_undegraded":"2024-05-11T01:01:45.306352+0000","last_fullsized":"2024-05-11T01:01:45.306352+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:49:08.675245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.274889+0000","last_change":"2024-05-11T01:01:44.285570+0000","last_active":"2024-05-11T01:01:45.274889+0000","last_peered":"2024-05-11T01:01:45.274889+0000","last_clean":"2024-05-11T01:01:45.274889+0000","last_became_active":"2024-05-11T01:01:44.285277+0000","last_became_peered":"2024-05-11T01:01:44.285277+0000","last_unstale":"2024-05-11T01:01:45.274889+0000","last_undegraded":"2024-05-11T01:01:45.274889+0000","last_fullsized":"2024-05-11T01:01:45.274889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:23.115690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308180+0000","last_change":"2024-05-11T01:01:45.295740+0000","last_active":"2024-05-11T01:01:46.308180+0000","last_peered":"2024-05-11T01:01:46.308180+0000","last_clean":"2024-05-11T01:01:46.308180+0000","last_became_active":"2024-05-11T01:01:45.295233+0000","last_became_peered":"2024-05-11T01:01:45.295233+0000","last_unstale":"2024-05-11T01:01:46.308180+0000","last_undegraded":"2024-05-11T01:01:46.308180+0000","last_fullsized":"2024-05-11T01:01:46.308180+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:33.401596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305543+0000","last_change":"2024-05-11T01:01:44.284493+0000","last_active":"2024-05-11T01:01:45.305543+0000","last_peered":"2024-05-11T01:01:45.305543+0000","last_clean":"2024-05-11T01:01:45.305543+0000","last_became_active":"2024-05-11T01:01:44.284192+0000","last_became_peered":"2024-05-11T01:01:44.284192+0000","last_unstale":"2024-05-11T01:01:45.305543+0000","last_undegraded":"2024-05-11T01:01:45.305543+0000","last_fullsized":"2024-05-11T01:01:45.305543+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:09:23.191937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.283182+0000","last_change":"2024-05-11T01:01:45.280417+0000","last_active":"2024-05-11T01:01:45.283182+0000","last_peered":"2024-05-11T01:01:45.283182+0000","last_clean":"2024-05-11T01:01:45.283182+0000","last_became_active":"2024-05-11T01:01:45.280111+0000","last_became_peered":"2024-05-11T01:01:45.280111+0000","last_unstale":"2024-05-11T01:01:45.283182+0000","last_undegraded":"2024-05-11T01:01:45.283182+0000","last_fullsized":"2024-05-11T01:01:45.283182+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:51.236037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.306100+0000","last_change":"2024-05-11T01:01:44.288373+0000","last_active":"2024-05-11T01:01:45.306100+0000","last_peered":"2024-05-11T01:01:45.306100+0000","last_clean":"2024-05-11T01:01:45.306100+0000","last_became_active":"2024-05-11T01:01:44.288055+0000","last_became_peered":"2024-05-11T01:01:44.288055+0000","last_unstale":"2024-05-11T01:01:45.306100+0000","last_undegraded":"2024-05-11T01:01:45.306100+0000","last_fullsized":"2024-05-11T01:01:45.306100+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:50.230455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.288866+0000","last_change":"2024-05-11T01:01:45.281226+0000","last_active":"2024-05-11T01:01:45.288866+0000","last_peered":"2024-05-11T01:01:45.288866+0000","last_clean":"2024-05-11T01:01:45.288866+0000","last_became_active":"2024-05-11T01:01:45.280950+0000","last_became_peered":"2024-05-11T01:01:45.280950+0000","last_unstale":"2024-05-11T01:01:45.288866+0000","last_undegraded":"2024-05-11T01:01:45.288866+0000","last_fullsized":"2024-05-11T01:01:45.288866+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:51:33.401596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.298906+0000","last_change":"2024-05-11T01:01:44.285957+0000","last_active":"2024-05-11T01:01:45.298906+0000","last_peered":"2024-05-11T01:01:45.298906+0000","last_clean":"2024-05-11T01:01:45.298906+0000","last_became_active":"2024-05-11T01:01:44.285672+0000","last_became_peered":"2024-05-11T01:01:44.285672+0000","last_unstale":"2024-05-11T01:01:45.298906+0000","last_undegraded":"2024-05-11T01:01:45.298906+0000","last_fullsized":"2024-05-11T01:01:45.298906+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:16.693227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.282633+0000","last_change":"2024-05-11T01:01:45.281528+0000","last_active":"2024-05-11T01:01:45.282633+0000","last_peered":"2024-05-11T01:01:45.282633+0000","last_clean":"2024-05-11T01:01:45.282633+0000","last_became_active":"2024-05-11T01:01:45.281191+0000","last_became_peered":"2024-05-11T01:01:45.281191+0000","last_unstale":"2024-05-11T01:01:45.282633+0000","last_undegraded":"2024-05-11T01:01:45.282633+0000","last_fullsized":"2024-05-11T01:01:45.282633+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:06.518581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301664+0000","last_change":"2024-05-11T01:01:44.281083+0000","last_active":"2024-05-11T01:01:45.301664+0000","last_peered":"2024-05-11T01:01:45.301664+0000","last_clean":"2024-05-11T01:01:45.301664+0000","last_became_active":"2024-05-11T01:01:44.280737+0000","last_became_peered":"2024-05-11T01:01:44.280737+0000","last_unstale":"2024-05-11T01:01:45.301664+0000","last_undegraded":"2024-05-11T01:01:45.301664+0000","last_fullsized":"2024-05-11T01:01:45.301664+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:24:37.529966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301438+0000","last_change":"2024-05-11T01:01:45.289836+0000","last_active":"2024-05-11T01:01:45.301438+0000","last_peered":"2024-05-11T01:01:45.301438+0000","last_clean":"2024-05-11T01:01:45.301438+0000","last_became_active":"2024-05-11T01:01:45.289511+0000","last_became_peered":"2024-05-11T01:01:45.289511+0000","last_unstale":"2024-05-11T01:01:45.301438+0000","last_undegraded":"2024-05-11T01:01:45.301438+0000","last_fullsized":"2024-05-11T01:01:45.301438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:36:37.828556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.278480+0000","last_change":"2024-05-11T01:01:44.288592+0000","last_active":"2024-05-11T01:01:45.278480+0000","last_peered":"2024-05-11T01:01:45.278480+0000","last_clean":"2024-05-11T01:01:45.278480+0000","last_became_active":"2024-05-11T01:01:44.288275+0000","last_became_peered":"2024-05-11T01:01:44.288275+0000","last_unstale":"2024-05-11T01:01:45.278480+0000","last_undegraded":"2024-05-11T01:01:45.278480+0000","last_fullsized":"2024-05-11T01:01:45.278480+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:23:10.377267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.288194+0000","last_change":"2024-05-11T01:01:45.282757+0000","last_active":"2024-05-11T01:01:45.288194+0000","last_peered":"2024-05-11T01:01:45.288194+0000","last_clean":"2024-05-11T01:01:45.288194+0000","last_became_active":"2024-05-11T01:01:45.281211+0000","last_became_peered":"2024-05-11T01:01:45.281211+0000","last_unstale":"2024-05-11T01:01:45.288194+0000","last_undegraded":"2024-05-11T01:01:45.288194+0000","last_fullsized":"2024-05-11T01:01:45.288194+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:27:08.460825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.307721+0000","last_change":"2024-05-11T01:01:44.288683+0000","last_active":"2024-05-11T01:01:45.307721+0000","last_peered":"2024-05-11T01:01:45.307721+0000","last_clean":"2024-05-11T01:01:45.307721+0000","last_became_active":"2024-05-11T01:01:44.288379+0000","last_became_peered":"2024-05-11T01:01:44.288379+0000","last_unstale":"2024-05-11T01:01:45.307721+0000","last_undegraded":"2024-05-11T01:01:45.307721+0000","last_fullsized":"2024-05-11T01:01:45.307721+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:24:37.529966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301370+0000","last_change":"2024-05-11T01:01:45.290037+0000","last_active":"2024-05-11T01:01:45.301370+0000","last_peered":"2024-05-11T01:01:45.301370+0000","last_clean":"2024-05-11T01:01:45.301370+0000","last_became_active":"2024-05-11T01:01:45.289765+0000","last_became_peered":"2024-05-11T01:01:45.289765+0000","last_unstale":"2024-05-11T01:01:45.301370+0000","last_undegraded":"2024-05-11T01:01:45.301370+0000","last_fullsized":"2024-05-11T01:01:45.301370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:46:16.460528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308214+0000","last_change":"2024-05-11T01:01:44.291362+0000","last_active":"2024-05-11T01:01:46.308214+0000","last_peered":"2024-05-11T01:01:46.308214+0000","last_clean":"2024-05-11T01:01:46.308214+0000","last_became_active":"2024-05-11T01:01:44.291003+0000","last_became_peered":"2024-05-11T01:01:44.291003+0000","last_unstale":"2024-05-11T01:01:46.308214+0000","last_undegraded":"2024-05-11T01:01:46.308214+0000","last_fullsized":"2024-05-11T01:01:46.308214+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:24:37.529966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301156+0000","last_change":"2024-05-11T01:01:45.290031+0000","last_active":"2024-05-11T01:01:45.301156+0000","last_peered":"2024-05-11T01:01:45.301156+0000","last_clean":"2024-05-11T01:01:45.301156+0000","last_became_active":"2024-05-11T01:01:45.289719+0000","last_became_peered":"2024-05-11T01:01:45.289719+0000","last_unstale":"2024-05-11T01:01:45.301156+0000","last_undegraded":"2024-05-11T01:01:45.301156+0000","last_fullsized":"2024-05-11T01:01:45.301156+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:31:01.490189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.278256+0000","last_change":"2024-05-11T01:01:44.291340+0000","last_active":"2024-05-11T01:01:45.278256+0000","last_peered":"2024-05-11T01:01:45.278256+0000","last_clean":"2024-05-11T01:01:45.278256+0000","last_became_active":"2024-05-11T01:01:44.291028+0000","last_became_peered":"2024-05-11T01:01:44.291028+0000","last_unstale":"2024-05-11T01:01:45.278256+0000","last_undegraded":"2024-05-11T01:01:45.278256+0000","last_fullsized":"2024-05-11T01:01:45.278256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:37:55.593960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.295915+0000","last_change":"2024-05-11T01:01:45.291784+0000","last_active":"2024-05-11T01:01:45.295915+0000","last_peered":"2024-05-11T01:01:45.295915+0000","last_clean":"2024-05-11T01:01:45.295915+0000","last_became_active":"2024-05-11T01:01:45.291431+0000","last_became_peered":"2024-05-11T01:01:45.291431+0000","last_unstale":"2024-05-11T01:01:45.295915+0000","last_undegraded":"2024-05-11T01:01:45.295915+0000","last_fullsized":"2024-05-11T01:01:45.295915+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:11:53.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.274817+0000","last_change":"2024-05-11T01:01:44.287271+0000","last_active":"2024-05-11T01:01:45.274817+0000","last_peered":"2024-05-11T01:01:45.274817+0000","last_clean":"2024-05-11T01:01:45.274817+0000","last_became_active":"2024-05-11T01:01:44.286962+0000","last_became_peered":"2024-05-11T01:01:44.286962+0000","last_unstale":"2024-05-11T01:01:45.274817+0000","last_undegraded":"2024-05-11T01:01:45.274817+0000","last_fullsized":"2024-05-11T01:01:45.274817+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:39.042079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301336+0000","last_change":"2024-05-11T01:01:45.292682+0000","last_active":"2024-05-11T01:01:45.301336+0000","last_peered":"2024-05-11T01:01:45.301336+0000","last_clean":"2024-05-11T01:01:45.301336+0000","last_became_active":"2024-05-11T01:01:45.292395+0000","last_became_peered":"2024-05-11T01:01:45.292395+0000","last_unstale":"2024-05-11T01:01:45.301336+0000","last_undegraded":"2024-05-11T01:01:45.301336+0000","last_fullsized":"2024-05-11T01:01:45.301336+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:49:15.492453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.301699+0000","last_change":"2024-05-11T01:01:44.291611+0000","last_active":"2024-05-11T01:01:45.301699+0000","last_peered":"2024-05-11T01:01:45.301699+0000","last_clean":"2024-05-11T01:01:45.301699+0000","last_became_active":"2024-05-11T01:01:44.291194+0000","last_became_peered":"2024-05-11T01:01:44.291194+0000","last_unstale":"2024-05-11T01:01:45.301699+0000","last_undegraded":"2024-05-11T01:01:45.301699+0000","last_fullsized":"2024-05-11T01:01:45.301699+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:23.115690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.289806+0000","last_change":"2024-05-11T01:01:45.285159+0000","last_active":"2024-05-11T01:01:45.289806+0000","last_peered":"2024-05-11T01:01:45.289806+0000","last_clean":"2024-05-11T01:01:45.289806+0000","last_became_active":"2024-05-11T01:01:45.284418+0000","last_became_peered":"2024-05-11T01:01:45.284418+0000","last_unstale":"2024-05-11T01:01:45.289806+0000","last_undegraded":"2024-05-11T01:01:45.289806+0000","last_fullsized":"2024-05-11T01:01:45.289806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:11:53.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.283749+0000","last_change":"2024-05-11T01:01:44.288773+0000","last_active":"2024-05-11T01:01:45.283749+0000","last_peered":"2024-05-11T01:01:45.283749+0000","last_clean":"2024-05-11T01:01:45.283749+0000","last_became_active":"2024-05-11T01:01:44.288477+0000","last_became_peered":"2024-05-11T01:01:44.288477+0000","last_unstale":"2024-05-11T01:01:45.283749+0000","last_undegraded":"2024-05-11T01:01:45.283749+0000","last_fullsized":"2024-05-11T01:01:45.283749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:21:16.693227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.291744+0000","last_change":"2024-05-11T01:01:45.289314+0000","last_active":"2024-05-11T01:01:45.291744+0000","last_peered":"2024-05-11T01:01:45.291744+0000","last_clean":"2024-05-11T01:01:45.291744+0000","last_became_active":"2024-05-11T01:01:45.288540+0000","last_became_peered":"2024-05-11T01:01:45.288540+0000","last_unstale":"2024-05-11T01:01:45.291744+0000","last_undegraded":"2024-05-11T01:01:45.291744+0000","last_fullsized":"2024-05-11T01:01:45.291744+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:30:51.236037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.275334+0000","last_change":"2024-05-11T01:01:44.288644+0000","last_active":"2024-05-11T01:01:45.275334+0000","last_peered":"2024-05-11T01:01:45.275334+0000","last_clean":"2024-05-11T01:01:45.275334+0000","last_became_active":"2024-05-11T01:01:44.288315+0000","last_became_peered":"2024-05-11T01:01:44.288315+0000","last_unstale":"2024-05-11T01:01:45.275334+0000","last_undegraded":"2024-05-11T01:01:45.275334+0000","last_fullsized":"2024-05-11T01:01:45.275334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:02:36.637341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.308113+0000","last_change":"2024-05-11T01:01:45.303836+0000","last_active":"2024-05-11T01:01:46.308113+0000","last_peered":"2024-05-11T01:01:46.308113+0000","last_clean":"2024-05-11T01:01:46.308113+0000","last_became_active":"2024-05-11T01:01:45.303521+0000","last_became_peered":"2024-05-11T01:01:45.303521+0000","last_unstale":"2024-05-11T01:01:46.308113+0000","last_undegraded":"2024-05-11T01:01:46.308113+0000","last_fullsized":"2024-05-11T01:01:46.308113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:40.047660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.285641+0000","last_change":"2024-05-11T01:01:44.284801+0000","last_active":"2024-05-11T01:01:45.285641+0000","last_peered":"2024-05-11T01:01:45.285641+0000","last_clean":"2024-05-11T01:01:45.285641+0000","last_became_active":"2024-05-11T01:01:44.284389+0000","last_became_peered":"2024-05-11T01:01:44.284389+0000","last_unstale":"2024-05-11T01:01:45.285641+0000","last_undegraded":"2024-05-11T01:01:45.285641+0000","last_fullsized":"2024-05-11T01:01:45.285641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:17:19.132665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.305297+0000","last_change":"2024-05-11T01:01:45.302950+0000","last_active":"2024-05-11T01:01:45.305297+0000","last_peered":"2024-05-11T01:01:45.305297+0000","last_clean":"2024-05-11T01:01:45.305297+0000","last_became_active":"2024-05-11T01:01:45.302359+0000","last_became_peered":"2024-05-11T01:01:45.302359+0000","last_unstale":"2024-05-11T01:01:45.305297+0000","last_undegraded":"2024-05-11T01:01:45.305297+0000","last_fullsized":"2024-05-11T01:01:45.305297+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:24.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.277701+0000","last_change":"2024-05-11T01:01:44.290395+0000","last_active":"2024-05-11T01:01:45.277701+0000","last_peered":"2024-05-11T01:01:45.277701+0000","last_clean":"2024-05-11T01:01:45.277701+0000","last_became_active":"2024-05-11T01:01:44.289930+0000","last_became_peered":"2024-05-11T01:01:44.289930+0000","last_unstale":"2024-05-11T01:01:45.277701+0000","last_undegraded":"2024-05-11T01:01:45.277701+0000","last_fullsized":"2024-05-11T01:01:45.277701+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:43:00.679498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.300594+0000","last_change":"2024-05-11T01:01:45.295756+0000","last_active":"2024-05-11T01:01:45.300594+0000","last_peered":"2024-05-11T01:01:45.300594+0000","last_clean":"2024-05-11T01:01:45.300594+0000","last_became_active":"2024-05-11T01:01:45.295261+0000","last_became_peered":"2024-05-11T01:01:45.295261+0000","last_unstale":"2024-05-11T01:01:45.300594+0000","last_undegraded":"2024-05-11T01:01:45.300594+0000","last_fullsized":"2024-05-11T01:01:45.300594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:14.953184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.278486+0000","last_change":"2024-05-11T01:01:44.282707+0000","last_active":"2024-05-11T01:01:45.278486+0000","last_peered":"2024-05-11T01:01:45.278486+0000","last_clean":"2024-05-11T01:01:45.278486+0000","last_became_active":"2024-05-11T01:01:44.282174+0000","last_became_peered":"2024-05-11T01:01:44.282174+0000","last_unstale":"2024-05-11T01:01:45.278486+0000","last_undegraded":"2024-05-11T01:01:45.278486+0000","last_fullsized":"2024-05-11T01:01:45.278486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:21:20.490794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.299361+0000","last_change":"2024-05-11T01:01:45.295924+0000","last_active":"2024-05-11T01:01:45.299361+0000","last_peered":"2024-05-11T01:01:45.299361+0000","last_clean":"2024-05-11T01:01:45.299361+0000","last_became_active":"2024-05-11T01:01:45.295072+0000","last_became_peered":"2024-05-11T01:01:45.295072+0000","last_unstale":"2024-05-11T01:01:45.299361+0000","last_undegraded":"2024-05-11T01:01:45.299361+0000","last_fullsized":"2024-05-11T01:01:45.299361+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:43:01.685079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.276462+0000","last_change":"2024-05-11T01:01:44.288428+0000","last_active":"2024-05-11T01:01:45.276462+0000","last_peered":"2024-05-11T01:01:45.276462+0000","last_clean":"2024-05-11T01:01:45.276462+0000","last_became_active":"2024-05-11T01:01:44.287474+0000","last_became_peered":"2024-05-11T01:01:44.287474+0000","last_unstale":"2024-05-11T01:01:45.276462+0000","last_undegraded":"2024-05-11T01:01:45.276462+0000","last_fullsized":"2024-05-11T01:01:45.276462+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:39.042079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.298940+0000","last_change":"2024-05-11T01:01:45.296282+0000","last_active":"2024-05-11T01:01:45.298940+0000","last_peered":"2024-05-11T01:01:45.298940+0000","last_clean":"2024-05-11T01:01:45.298940+0000","last_became_active":"2024-05-11T01:01:45.296019+0000","last_became_peered":"2024-05-11T01:01:45.296019+0000","last_unstale":"2024-05-11T01:01:45.298940+0000","last_undegraded":"2024-05-11T01:01:45.298940+0000","last_fullsized":"2024-05-11T01:01:45.298940+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:44.263900+0000","last_clean_scrub_stamp":"2024-05-11T01:01:44.263900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:23:44.478340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-11T01:01:46.307691+0000","last_change":"2024-05-11T01:01:44.284268+0000","last_active":"2024-05-11T01:01:46.307691+0000","last_peered":"2024-05-11T01:01:46.307691+0000","last_clean":"2024-05-11T01:01:46.307691+0000","last_became_active":"2024-05-11T01:01:44.283918+0000","last_became_peered":"2024-05-11T01:01:44.283918+0000","last_unstale":"2024-05-11T01:01:46.307691+0000","last_undegraded":"2024-05-11T01:01:46.307691+0000","last_fullsized":"2024-05-11T01:01:46.307691+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:01:43.258319+0000","last_clean_scrub_stamp":"2024-05-11T01:01:43.258319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:17:19.132665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":127,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-11T01:01:45.282110+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:01:45.282110+0000","last_peered":"2024-05-11T01:01:45.282110+0000","last_clean":"2024-05-11T01:01:45.282110+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:01:45.282110+0000","last_undegraded":"2024-05-11T01:01:45.282110+0000","last_fullsized":"2024-05-11T01:01:45.282110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":128162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":711139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":128218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":127650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":129180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31500,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340340,"statfs":{"total":96636764160,"available":96604508160,"internally_reserved":0,"allocated":323584,"data_stored":120771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31500,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340340,"statfs":{"total":96636764160,"available":96604508160,"internally_reserved":0,"allocated":323584,"data_stored":120771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":711139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:01:49.704 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T01:01:49.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-11T01:01:49.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.851+0000 7f68e3493640 1 Processor -- start 2024-05-11T01:01:49.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.851+0000 7f68e3493640 1 -- start start 2024-05-11T01:01:49.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.852+0000 7f68e3493640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68dc0d4e40 0x7f68dc0d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.852+0000 7f68e3493640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc0d2d60 0x7f68dc0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.852+0000 7f68e3493640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68dc0d3a90 con 0x7f68dc15f2f0 2024-05-11T01:01:49.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.852+0000 7f68e3493640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f68dc0d3d70 con 0x7f68dc0d4e40 2024-05-11T01:01:49.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.852+0000 7f68e3493640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc0d4050 con 0x7f68dc0d2d60 2024-05-11T01:01:49.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.853+0000 7f68e1a30640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc0d2d60 0x7f68dc0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.853+0000 7f68e0a2e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68dc0d4e40 0x7f68dc0d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.853+0000 7f68e122f640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f68dc15f2f0 0x7f68dc0cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:38748/0 (socket says 172.21.15.52:38748) 2024-05-11T01:01:49.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.853+0000 7f68e122f640 1 -- 172.21.15.52:0/2412749475 learned_addr learned my addr 172.21.15.52:0/2412749475 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:49.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.853+0000 7f68e1a30640 1 -- 172.21.15.52:0/2412749475 >> v2:172.21.15.119:3300/0 conn(0x7f68dc0d4e40 msgr2=0x7f68dc0d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:49.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.853+0000 7f68e1a30640 1 --2- 172.21.15.52:0/2412749475 >> v2:172.21.15.119:3300/0 conn(0x7f68dc0d4e40 0x7f68dc0d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.853+0000 7f68e1a30640 1 -- 172.21.15.52:0/2412749475 >> v1:172.21.15.119:6789/0 conn(0x7f68dc15f2f0 legacy=0x7f68dc0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.853+0000 7f68e1a30640 1 -- 172.21.15.52:0/2412749475 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc0b4e60 con 0x7f68dc0d2d60 2024-05-11T01:01:49.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68e1a30640 1 --2- 172.21.15.52:0/2412749475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc0d2d60 0x7f68dc0d31e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009d90 tx=0x7f68d402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6467c27b9e85196f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:49.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68d3fff640 1 -- 172.21.15.52:0/2412749475 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4007740 con 0x7f68dc0d2d60 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68d3fff640 1 -- 172.21.15.52:0/2412749475 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d40078e0 con 0x7f68dc0d2d60 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68d3fff640 1 -- 172.21.15.52:0/2412749475 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4007cb0 con 0x7f68dc0d2d60 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68e3493640 1 -- 172.21.15.52:0/2412749475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc0d2d60 msgr2=0x7f68dc0d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68e3493640 1 --2- 172.21.15.52:0/2412749475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc0d2d60 0x7f68dc0d31e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009d90 tx=0x7f68d402a660 comp rx=0 tx=0).stop 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68e3493640 1 -- 172.21.15.52:0/2412749475 shutdown_connections 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68e3493640 1 --2- 172.21.15.52:0/2412749475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc0d2d60 0x7f68dc0d31e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68e3493640 1 --2- 172.21.15.52:0/2412749475 >> v2:172.21.15.119:3300/0 conn(0x7f68dc0d4e40 0x7f68dc0d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68e3493640 1 -- 172.21.15.52:0/2412749475 >> 172.21.15.52:0/2412749475 conn(0x7f68dc08ebb0 msgr2=0x7f68dc09e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.854+0000 7f68e3493640 1 -- 172.21.15.52:0/2412749475 shutdown_connections 2024-05-11T01:01:49.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.855+0000 7f68e3493640 1 -- 172.21.15.52:0/2412749475 wait complete. 2024-05-11T01:01:49.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.856+0000 7f68e3493640 1 Processor -- start 2024-05-11T01:01:49.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.856+0000 7f68e3493640 1 -- start start 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.856+0000 7f68e3493640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68dc0d2d60 0x7f68dc0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.856+0000 7f68e3493640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68dc0d4e40 0x7f68dc150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.856+0000 7f68e3493640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc15f2f0 0x7f68dc06dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.856+0000 7f68e3493640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc0d3720 con 0x7f68dc0d2d60 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.856+0000 7f68e3493640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc0d57d0 con 0x7f68dc0d4e40 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.856+0000 7f68e3493640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc15f6d0 con 0x7f68dc15f2f0 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.856+0000 7f68e122f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68dc0d2d60 0x7f68dc0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e122f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68dc0d2d60 0x7f68dc0ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40860/0 (socket says 172.21.15.52:40860) 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e122f640 1 -- 172.21.15.52:0/2408647129 learned_addr learned my addr 172.21.15.52:0/2408647129 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e0a2e640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68dc0d4e40 0x7f68dc150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e1a30640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc15f2f0 0x7f68dc06dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e122f640 1 -- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc15f2f0 msgr2=0x7f68dc06dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e122f640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc15f2f0 0x7f68dc06dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e122f640 1 -- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68dc0d4e40 msgr2=0x7f68dc150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:49.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e122f640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68dc0d4e40 0x7f68dc150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:49.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e122f640 1 -- 172.21.15.52:0/2408647129 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc0c8da0 con 0x7f68dc0d2d60 2024-05-11T01:01:49.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e0a2e640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68dc0d4e40 0x7f68dc150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:49.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e122f640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68dc0d2d60 0x7f68dc0ce660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f68d8001eb0 tx=0x7f68d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:49.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68d1ffb640 1 -- 172.21.15.52:0/2408647129 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8017070 con 0x7f68dc0d2d60 2024-05-11T01:01:49.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68d1ffb640 1 -- 172.21.15.52:0/2408647129 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d8005d00 con 0x7f68dc0d2d60 2024-05-11T01:01:49.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc0ceeb0 con 0x7f68dc0d2d60 2024-05-11T01:01:49.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68d1ffb640 1 -- 172.21.15.52:0/2408647129 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8007310 con 0x7f68dc0d2d60 2024-05-11T01:01:49.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.857+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc0c9020 con 0x7f68dc0d2d60 2024-05-11T01:01:49.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.859+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68dc069f50 con 0x7f68dc0d2d60 2024-05-11T01:01:49.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.859+0000 7f68d1ffb640 1 -- 172.21.15.52:0/2408647129 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f68d801a070 con 0x7f68dc0d2d60 2024-05-11T01:01:49.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.860+0000 7f68d1ffb640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f68c00762d0 0x7f68c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:49.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.860+0000 7f68d1ffb640 1 -- 172.21.15.52:0/2408647129 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f68d80969d0 con 0x7f68dc0d2d60 2024-05-11T01:01:49.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.860+0000 7f68e0a2e640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f68c00762d0 0x7f68c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:49.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.860+0000 7f68e0a2e640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f68c00762d0 0x7f68c0078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68dc1f8970 tx=0x7f68cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:49.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:49.864+0000 7f68d1ffb640 1 -- 172.21.15.52:0/2408647129 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f68dc069f50 con 0x7f68dc0d2d60 2024-05-11T01:01:50.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.129+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f68dc0695d0 con 0x7f68dc0d2d60 2024-05-11T01:01:50.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.131+0000 7f68d1ffb640 1 -- 172.21.15.52:0/2408647129 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f68dc0c8da0 con 0x7f68dc0d2d60 2024-05-11T01:01:50.131 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-05-11T01:01:50.131 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5mUO0yKhAAfS2pcT6nSB+NrJccryMO8A== 2024-05-11T01:01:50.132 INFO:teuthology.orchestra.run.smithi052.stdout: caps mds = "allow" 2024-05-11T01:01:50.132 INFO:teuthology.orchestra.run.smithi052.stdout: caps mon = "allow r" 2024-05-11T01:01:50.132 INFO:teuthology.orchestra.run.smithi052.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-11T01:01:50.132 INFO:teuthology.orchestra.run.smithi052.stderr:updated caps for client.0 2024-05-11T01:01:50.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f68c00762d0 msgr2=0x7f68c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:50.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f68c00762d0 0x7f68c0078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68dc1f8970 tx=0x7f68cc008040 comp rx=0 tx=0).stop 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68dc0d2d60 msgr2=0x7f68dc0ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68dc0d2d60 0x7f68dc0ce660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f68d8001eb0 tx=0x7f68d800c890 comp rx=0 tx=0).stop 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 shutdown_connections 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f68c00762d0 0x7f68c0078790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68dc15f2f0 0x7f68dc06dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68dc0d4e40 0x7f68dc150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 --2- 172.21.15.52:0/2408647129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68dc0d2d60 0x7f68dc0ce660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 >> 172.21.15.52:0/2408647129 conn(0x7f68dc08ebb0 msgr2=0x7f68dc15d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 shutdown_connections 2024-05-11T01:01:50.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.133+0000 7f68e3493640 1 -- 172.21.15.52:0/2408647129 wait complete. 2024-05-11T01:01:50.144 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:01:50.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.248+0000 7fe2241af640 1 Processor -- start 2024-05-11T01:01:50.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.248+0000 7fe2241af640 1 -- start start 2024-05-11T01:01:50.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe2241af640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c159980 0x7fe21c162ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:50.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe2241af640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe21c06fbc0 0x7fe21c163420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:50.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe2241af640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe21c06c480 0x7fe21c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:50.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe2241af640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe21c159d60 con 0x7fe21c06fbc0 2024-05-11T01:01:50.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe2241af640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe21c0b6f90 con 0x7fe21c06c480 2024-05-11T01:01:50.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe2241af640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe21c163d30 con 0x7fe21c159980 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22174a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe21c06fbc0 0x7fe21c163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22174a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe21c06fbc0 0x7fe21c163420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40876/0 (socket says 172.21.15.52:40876) 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22174a640 1 -- 172.21.15.52:0/2277574266 learned_addr learned my addr 172.21.15.52:0/2277574266 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22174a640 1 -- 172.21.15.52:0/2277574266 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c159980 msgr2=0x7fe21c162ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe221f4b640 1 --2- 172.21.15.52:0/2277574266 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c159980 0x7fe21c162ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22274c640 1 --2- 172.21.15.52:0/2277574266 >> v2:172.21.15.119:3300/0 conn(0x7fe21c06c480 0x7fe21c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22174a640 1 --2- 172.21.15.52:0/2277574266 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c159980 0x7fe21c162ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22174a640 1 -- 172.21.15.52:0/2277574266 >> v2:172.21.15.119:3300/0 conn(0x7fe21c06c480 msgr2=0x7fe21c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22174a640 1 --2- 172.21.15.52:0/2277574266 >> v2:172.21.15.119:3300/0 conn(0x7fe21c06c480 0x7fe21c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22174a640 1 -- 172.21.15.52:0/2277574266 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21c163a50 con 0x7fe21c06fbc0 2024-05-11T01:01:50.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.249+0000 7fe22274c640 1 --2- 172.21.15.52:0/2277574266 >> v2:172.21.15.119:3300/0 conn(0x7fe21c06c480 0x7fe21c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:50.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.250+0000 7fe22174a640 1 --2- 172.21.15.52:0/2277574266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06fbc0 0x7fe21c163420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe21c163960 tx=0x7fe21002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc38b5c60d803fd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:50.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.250+0000 7fe220f49640 1 -- 172.21.15.52:0/2277574266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210004030 con 0x7fe21c06fbc0 2024-05-11T01:01:50.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.250+0000 7fe220f49640 1 -- 172.21.15.52:0/2277574266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2100041d0 con 0x7fe21c06fbc0 2024-05-11T01:01:50.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.250+0000 7fe220f49640 1 -- 172.21.15.52:0/2277574266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210003610 con 0x7fe21c06fbc0 2024-05-11T01:01:50.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.251+0000 7fe2241af640 1 -- 172.21.15.52:0/2277574266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06fbc0 msgr2=0x7fe21c163420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:50.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.251+0000 7fe2241af640 1 --2- 172.21.15.52:0/2277574266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06fbc0 0x7fe21c163420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe21c163960 tx=0x7fe21002a1c0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.251+0000 7fe2241af640 1 -- 172.21.15.52:0/2277574266 shutdown_connections 2024-05-11T01:01:50.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.251+0000 7fe2241af640 1 --2- 172.21.15.52:0/2277574266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06fbc0 0x7fe21c163420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.251+0000 7fe2241af640 1 --2- 172.21.15.52:0/2277574266 >> v2:172.21.15.119:3300/0 conn(0x7fe21c06c480 0x7fe21c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.251+0000 7fe2241af640 1 --2- 172.21.15.52:0/2277574266 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c159980 0x7fe21c162ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.251+0000 7fe2241af640 1 -- 172.21.15.52:0/2277574266 >> 172.21.15.52:0/2277574266 conn(0x7fe21c0b3af0 msgr2=0x7fe21c0b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:50.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.251+0000 7fe2241af640 1 -- 172.21.15.52:0/2277574266 shutdown_connections 2024-05-11T01:01:50.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.251+0000 7fe2241af640 1 -- 172.21.15.52:0/2277574266 wait complete. 2024-05-11T01:01:50.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.252+0000 7fe2241af640 1 Processor -- start 2024-05-11T01:01:50.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.252+0000 7fe2241af640 1 -- start start 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe2241af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06c480 0x7fe21c1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe2241af640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c06fbc0 0x7fe21c146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe2241af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c159980 0x7fe21c15af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe2241af640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c06f6a0 con 0x7fe21c06c480 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe2241af640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c06f3c0 con 0x7fe21c159980 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe2241af640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe21c0702f0 con 0x7fe21c06fbc0 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe221f4b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06c480 0x7fe21c1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe221f4b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06c480 0x7fe21c1459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40884/0 (socket says 172.21.15.52:40884) 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe221f4b640 1 -- 172.21.15.52:0/3945849684 learned_addr learned my addr 172.21.15.52:0/3945849684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:50.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe22274c640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c159980 0x7fe21c15af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe22174a640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c06fbc0 0x7fe21c146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe221f4b640 1 -- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c06fbc0 msgr2=0x7fe21c146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe221f4b640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c06fbc0 0x7fe21c146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe221f4b640 1 -- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c159980 msgr2=0x7fe21c15af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe221f4b640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c159980 0x7fe21c15af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe221f4b640 1 -- 172.21.15.52:0/3945849684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21c159ee0 con 0x7fe21c06c480 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe221f4b640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06c480 0x7fe21c1459b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c820 tx=0x7fe21400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe21a7fc640 1 -- 172.21.15.52:0/3945849684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214015070 con 0x7fe21c06c480 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.253+0000 7fe21a7fc640 1 -- 172.21.15.52:0/3945849684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe214004020 con 0x7fe21c06c480 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.254+0000 7fe21a7fc640 1 -- 172.21.15.52:0/3945849684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140043f0 con 0x7fe21c06c480 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.254+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21c15a180 con 0x7fe21c06c480 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.254+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe21c15a740 con 0x7fe21c06c480 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.254+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21c069780 con 0x7fe21c06c480 2024-05-11T01:01:50.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.255+0000 7fe21a7fc640 1 -- 172.21.15.52:0/3945849684 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe2140077d0 con 0x7fe21c06c480 2024-05-11T01:01:50.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.255+0000 7fe21a7fc640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe2080762d0 0x7fe208078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:50.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.256+0000 7fe21a7fc640 1 -- 172.21.15.52:0/3945849684 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fe214097290 con 0x7fe21c06c480 2024-05-11T01:01:50.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.256+0000 7fe22174a640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe2080762d0 0x7fe208078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:50.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.256+0000 7fe22174a640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe2080762d0 0x7fe208078790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe210033000 tx=0x7fe21002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.260+0000 7fe21a7fc640 1 -- 172.21.15.52:0/3945849684 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe2140605a0 con 0x7fe21c06c480 2024-05-11T01:01:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.492+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe21c06ca00 con 0x7fe21c06c480 2024-05-11T01:01:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.492+0000 7fe21a7fc640 1 -- 172.21.15.52:0/3945849684 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7fe2140648a0 con 0x7fe21c06c480 2024-05-11T01:01:50.492 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:01:50.492 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3122737368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3122737368},{"type":"v1","addr":"172.21.15.119:6836","nonce":3122737368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/211602681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":211602681},{"type":"v1","addr":"172.21.15.119:6837","nonce":211602681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2079689543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2079689543},{"type":"v1","addr":"172.21.15.52:6835","nonce":2079689543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:44.660501+0000","modified":"2024-05-11T01:01:47.669355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4854":{"gid":4854,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1325827549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1325827549},{"type":"v1","addr":"172.21.15.52:6834","nonce":1325827549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4854,"qdb_cluster":[4854]},"id":2}]} 2024-05-11T01:01:50.493 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.494+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe2080762d0 msgr2=0x7fe208078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.494+0000 7fe2241af640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe2080762d0 0x7fe208078790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe210033000 tx=0x7fe21002e000 comp rx=0 tx=0).stop 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.494+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06c480 msgr2=0x7fe21c1459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.494+0000 7fe2241af640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06c480 0x7fe21c1459b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c820 tx=0x7fe21400cce0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.494+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 shutdown_connections 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.494+0000 7fe2241af640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe2080762d0 0x7fe208078790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.494+0000 7fe2241af640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c159980 0x7fe21c15af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.494+0000 7fe2241af640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe21c06fbc0 0x7fe21c146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.494+0000 7fe2241af640 1 --2- 172.21.15.52:0/3945849684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe21c06c480 0x7fe21c1459b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.495+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 >> 172.21.15.52:0/3945849684 conn(0x7fe21c0b3af0 msgr2=0x7fe21c0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.495+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 shutdown_connections 2024-05-11T01:01:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:50.495+0000 7fe2241af640 1 -- 172.21.15.52:0/3945849684 wait complete. 2024-05-11T01:01:50.908 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T01:01:44.660501+0000', 'modified': '2024-05-11T01:01:47.669355+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4854}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4854': {'gid': 4854, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6834/1325827549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6832', 'nonce': 1325827549}, {'type': 'v1', 'addr': '172.21.15.52:6834', 'nonce': 1325827549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4854, 'qdb_cluster': [4854]} 2024-05-11T01:01:50.908 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T01:01:50.909 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-11T01:01:51.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.020+0000 7f2923390640 1 Processor -- start 2024-05-11T01:01:51.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.020+0000 7f2923390640 1 -- start start 2024-05-11T01:01:51.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.021+0000 7f2923390640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0b5dd0 0x7f291c0d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.021+0000 7f2923390640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291c163250 con 0x7f291c15f430 2024-05-11T01:01:51.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.021+0000 7f2923390640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291c163530 con 0x7f291c160480 2024-05-11T01:01:51.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.021+0000 7f2923390640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f291c163810 con 0x7f291c0b5dd0 2024-05-11T01:01:51.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.021+0000 7f292112c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f291c15f430 0x7f291c15fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33882/0 (socket says 172.21.15.52:33882) 2024-05-11T01:01:51.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.021+0000 7f292112c640 1 -- 172.21.15.52:0/2634191233 learned_addr learned my addr 172.21.15.52:0/2634191233 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:51.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718548233 0 0) 0x7f291c163250 con 0x7f291c15f430 2024-05-11T01:01:51.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2908003520 con 0x7f291c15f430 2024-05-11T01:01:51.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292192d640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0b5dd0 0x7f291c0d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4107260117 0 0) 0x7f2908003520 con 0x7f291c15f430 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0b5dd0 msgr2=0x7f291c0d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0b5dd0 0x7f291c0d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 >> v1:172.21.15.119:6789/0 conn(0x7f291c160480 legacy=0x7f291c0d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c163af0 con 0x7f291c15f430 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f2918001a90 con 0x7f291c15f430 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 >> v1:172.21.15.52:6789/0 conn(0x7f291c15f430 legacy=0x7f291c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2908003a30 0x7f2908003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2908004800 0x7f2908004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29080052e0 0x7f290800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f291c163810 con 0x7f2908003a30 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f290800c110 con 0x7f2908004800 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f290800c310 con 0x7f29080052e0 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292192d640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0b5dd0 0x7f291c0d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292112c640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2908003a30 0x7f2908003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292092b640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2908004800 0x7f2908004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292192d640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29080052e0 0x7f290800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292112c640 1 -- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29080052e0 msgr2=0x7f290800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292112c640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29080052e0 0x7f290800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292112c640 1 -- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2908004800 msgr2=0x7f2908004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292112c640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2908004800 0x7f2908004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.022+0000 7f292112c640 1 -- 172.21.15.52:0/2634191233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2908003520 con 0x7f2908003a30 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.023+0000 7f292112c640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2908003a30 0x7f2908003e50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f291800a890 tx=0x7f2908005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458356ef4d74bbfc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.023+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291800a4e0 con 0x7f2908003a30 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.023+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f291802f070 con 0x7f2908003a30 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.023+0000 7f2913fff640 1 -- 172.21.15.52:0/2634191233 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291802bbf0 con 0x7f2908003a30 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.024+0000 7f2923390640 1 -- 172.21.15.52:0/2634191233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2908003a30 msgr2=0x7f2908003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.024+0000 7f2923390640 1 --2- 172.21.15.52:0/2634191233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2908003a30 0x7f2908003e50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f291800a890 tx=0x7f2908005830 comp rx=0 tx=0).stop 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.024+0000 7f292192d640 1 -- 172.21.15.52:0/2634191233 reap_dead start 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.024+0000 7f2923390640 1 -- 172.21.15.52:0/2634191233 shutdown_connections 2024-05-11T01:01:51.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.024+0000 7f2923390640 1 -- 172.21.15.52:0/2634191233 >> 172.21.15.52:0/2634191233 conn(0x7f291c08eb70 msgr2=0x7f291c09e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:51.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.024+0000 7f2923390640 1 -- 172.21.15.52:0/2634191233 shutdown_connections 2024-05-11T01:01:51.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.024+0000 7f2923390640 1 -- 172.21.15.52:0/2634191233 wait complete. 2024-05-11T01:01:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.025+0000 7f2923390640 1 Processor -- start 2024-05-11T01:01:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.025+0000 7f2923390640 1 -- start start 2024-05-11T01:01:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f2923390640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2908003a30 0x7f291c0d3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f2923390640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f291c14ddf0 0x7f291c14e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f2923390640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0cd260 0x7f291c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f2923390640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f291c163030 con 0x7f291c14ddf0 2024-05-11T01:01:51.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f2923390640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f291c162eb0 con 0x7f2908003a30 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f2923390640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f291c1608f0 con 0x7f291c0cd260 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292092b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f291c14ddf0 0x7f291c14e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292092b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f291c14ddf0 0x7f291c14e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40892/0 (socket says 172.21.15.52:40892) 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292092b640 1 -- 172.21.15.52:0/2196099038 learned_addr learned my addr 172.21.15.52:0/2196099038 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292112c640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2908003a30 0x7f291c0d3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292192d640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0cd260 0x7f291c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292092b640 1 -- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0cd260 msgr2=0x7f291c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292092b640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0cd260 0x7f291c0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292092b640 1 -- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2908003a30 msgr2=0x7f291c0d3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292092b640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2908003a30 0x7f291c0d3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.026+0000 7f292092b640 1 -- 172.21.15.52:0/2196099038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c14e790 con 0x7f291c14ddf0 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.027+0000 7f292092b640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f291c14ddf0 0x7f291c14e250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f290c00f820 tx=0x7f290c00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.027+0000 7f2911ffb640 1 -- 172.21.15.52:0/2196099038 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c0050d0 con 0x7f291c14ddf0 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.027+0000 7f2911ffb640 1 -- 172.21.15.52:0/2196099038 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f290c018070 con 0x7f291c14ddf0 2024-05-11T01:01:51.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.027+0000 7f2923390640 1 -- 172.21.15.52:0/2196099038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c14ea00 con 0x7f291c14ddf0 2024-05-11T01:01:51.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.027+0000 7f2911ffb640 1 -- 172.21.15.52:0/2196099038 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c005490 con 0x7f291c14ddf0 2024-05-11T01:01:51.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.027+0000 7f2923390640 1 -- 172.21.15.52:0/2196099038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c14be00 con 0x7f291c14ddf0 2024-05-11T01:01:51.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.029+0000 7f2911ffb640 1 -- 172.21.15.52:0/2196099038 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f290c00d730 con 0x7f291c14ddf0 2024-05-11T01:01:51.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.029+0000 7f2911ffb640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29000762d0 0x7f2900078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.029+0000 7f2911ffb640 1 -- 172.21.15.52:0/2196099038 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f290c097540 con 0x7f291c14ddf0 2024-05-11T01:01:51.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.029+0000 7f292112c640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29000762d0 0x7f2900078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.030+0000 7f292112c640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29000762d0 0x7f2900078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2918033000 tx=0x7f2918007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.030+0000 7f2926185740 1 Processor -- start 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.030+0000 7f2926185740 1 -- start start 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f2926185740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a3d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f2926185740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f2926185740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a0dc50 0x55ff11a11460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f2926185740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ff119778d0 con 0x55ff11a3aff0 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f2926185740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ff11930c10 con 0x55ff11a3b3d0 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f2926185740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x55ff11973890 con 0x55ff11a0dc50 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292112c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a3d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292112c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a3d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40894/0 (socket says 172.21.15.52:40894) 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292112c640 1 -- 172.21.15.52:0/3326721055 learned_addr learned my addr 172.21.15.52:0/3326721055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292192d640 1 --2- 172.21.15.52:0/3326721055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a0dc50 0x55ff11a11460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292092b640 1 --2- 172.21.15.52:0/3326721055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292112c640 1 -- 172.21.15.52:0/3326721055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a0dc50 msgr2=0x55ff11a11460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292112c640 1 --2- 172.21.15.52:0/3326721055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a0dc50 0x55ff11a11460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292112c640 1 -- 172.21.15.52:0/3326721055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 msgr2=0x55ff11a3dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292112c640 1 --2- 172.21.15.52:0/3326721055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292112c640 1 -- 172.21.15.52:0/3326721055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff119385d0 con 0x55ff11a3aff0 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f292112c640 1 --2- 172.21.15.52:0/3326721055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a3d800 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f291802b4a0 tx=0x7f291802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81245ceaf0112b4b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f2906ffd640 1 -- 172.21.15.52:0/3326721055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291802bc40 con 0x55ff11a3aff0 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f2906ffd640 1 -- 172.21.15.52:0/3326721055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f291802f040 con 0x55ff11a3aff0 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.031+0000 7f2906ffd640 1 -- 172.21.15.52:0/3326721055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918034940 con 0x55ff11a3aff0 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.032+0000 7f2926185740 1 -- 172.21.15.52:0/3326721055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 msgr2=0x55ff11a3d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.032+0000 7f2926185740 1 --2- 172.21.15.52:0/3326721055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a3d800 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f291802b4a0 tx=0x7f291802d000 comp rx=0 tx=0).stop 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.032+0000 7f2926185740 1 -- 172.21.15.52:0/3326721055 shutdown_connections 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.032+0000 7f2926185740 1 --2- 172.21.15.52:0/3326721055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a0dc50 0x55ff11a11460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.032+0000 7f2926185740 1 --2- 172.21.15.52:0/3326721055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.032+0000 7f2926185740 1 --2- 172.21.15.52:0/3326721055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a3d800 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.032+0000 7f2926185740 1 -- 172.21.15.52:0/3326721055 >> 172.21.15.52:0/3326721055 conn(0x55ff11a0fec0 msgr2=0x55ff11a58800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:51.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.032+0000 7f2926185740 1 -- 172.21.15.52:0/3326721055 shutdown_connections 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.032+0000 7f2926185740 1 -- 172.21.15.52:0/3326721055 wait complete. 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.033+0000 7f2926185740 1 -- start start 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.033+0000 7f2926185740 1 Processor -- start 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f2926185740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f2926185740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 0x55ff11a54030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f2926185740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 0x55ff11a5aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f2926185740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ff11a13070 con 0x55ff11a3aff0 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f2926185740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ff11934760 con 0x55ff11a3b3d0 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f2926185740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x55ff11a13350 con 0x55ff11a54570 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f2926185740 10 client.0 _ll_register_callbacks cb 0x55ff1196dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292112c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292112c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40896/0 (socket says 172.21.15.52:40896) 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292112c640 1 -- 172.21.15.52:0/2228661423 learned_addr learned my addr 172.21.15.52:0/2228661423 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292192d640 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 0x55ff11a5aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292092b640 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 0x55ff11a54030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292112c640 1 -- 172.21.15.52:0/2228661423 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 msgr2=0x55ff11a5aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292112c640 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 0x55ff11a5aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292112c640 1 -- 172.21.15.52:0/2228661423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 msgr2=0x55ff11a54030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292112c640 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 0x55ff11a54030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292112c640 1 -- 172.21.15.52:0/2228661423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff116892d0 con 0x55ff11a3aff0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f292112c640 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a538d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f291800a980 tx=0x7f291802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6624b0e44ee6a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f28fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f28fbfff640 1 -- 172.21.15.52:0/2228661423 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918055060 con 0x55ff11a3aff0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f28fbfff640 1 -- 172.21.15.52:0/2228661423 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f291804d460 con 0x55ff11a3aff0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.034+0000 7f2926185740 1 -- 172.21.15.52:0/2228661423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f291802f040 con 0x55ff11a3aff0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f28fbfff640 1 -- 172.21.15.52:0/2228661423 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f291805b050 con 0x55ff11a3aff0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f2926185740 10 client.5013 fetch_fsmap learned FSMap version 17 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f2926185740 1 -- 172.21.15.52:0/2228661423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55ff117536b0 con 0x55ff11a3aff0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f28fbfff640 1 -- 172.21.15.52:0/2228661423 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f291802f040 con 0x55ff11a3aff0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f28fbfff640 1 -- 172.21.15.52:0/2228661423 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f291805e020 con 0x55ff11a3aff0 2024-05-11T01:01:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f2926185740 10 client.5013 fetch_fsmap finished waiting for FSMap version 17 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f2926185740 10 client.5013 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f2926185740 20 client.5013 populate_metadata read hostname 'smithi052' 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f2926185740 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] conn(0x55ff11a6c290 0x55ff11a6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f2926185740 4 client.5013 mds_command: new command op to 4854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f2926185740 1 -- 172.21.15.52:0/2228661423 --> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff118fb760 con 0x55ff11a6c290 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f292092b640 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] conn(0x55ff11a6c290 0x55ff11a6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f28fbfff640 10 client.5013 ms_handle_connect on v2:172.21.15.52:6832/1325827549 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.035+0000 7f292092b640 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] conn(0x55ff11a6c290 0x55ff11a6c670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e6ae1b99c68e5a0 server_cookie=d9a083110b7b05a2 in_seq=0 out_seq=0 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f28fbfff640 1 -- 172.21.15.52:0/2228661423 <== mds.0 v2:172.21.15.52:6832/1325827549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff118fb760 con 0x55ff11a6c290 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f28fbfff640 10 client.5013 handle_command_reply: tid=0 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 client.5013 shutdown 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 -- 172.21.15.52:0/2228661423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 msgr2=0x55ff11a538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a538d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f291800a980 tx=0x7f291802d070 comp rx=0 tx=0).stop 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 -- 172.21.15.52:0/2228661423 shutdown_connections 2024-05-11T01:01:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] conn(0x55ff11a6c290 0x55ff11a6c670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 0x55ff11a5aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3b3d0 0x55ff11a54030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 --2- 172.21.15.52:0/2228661423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3aff0 0x55ff11a538d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f28fbfff640 0 client.5013 ms_handle_reset on v2:172.21.15.52:6832/1325827549 2024-05-11T01:01:51.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 -- 172.21.15.52:0/2228661423 >> 172.21.15.52:0/2228661423 conn(0x55ff11a0dc50 msgr2=0x55ff11a0eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:51.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 -- 172.21.15.52:0/2228661423 shutdown_connections 2024-05-11T01:01:51.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 1 -- 172.21.15.52:0/2228661423 wait complete. 2024-05-11T01:01:51.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.036+0000 7f2926185740 20 client.5013 trim_cache size 0 max 16384 2024-05-11T01:01:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.055+0000 7f2926185740 1 Processor -- start 2024-05-11T01:01:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.055+0000 7f2926185740 1 -- start start 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f2926185740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3aff0 0x55ff11a7c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f2926185740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a72860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f2926185740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 0x55ff11a11650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f2926185740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f291c2011a0 con 0x55ff11a3b3d0 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f2926185740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f291c14d430 con 0x55ff11a3aff0 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f2926185740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f291c14da80 con 0x55ff11a54570 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292092b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a72860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292092b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a72860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40908/0 (socket says 172.21.15.52:40908) 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292092b640 1 -- 172.21.15.52:0/596780750 learned_addr learned my addr 172.21.15.52:0/596780750 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292192d640 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 0x55ff11a11650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292112c640 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3aff0 0x55ff11a7c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292092b640 1 -- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 msgr2=0x55ff11a11650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292092b640 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 0x55ff11a11650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292092b640 1 -- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3aff0 msgr2=0x55ff11a7c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292092b640 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3aff0 0x55ff11a7c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292092b640 1 -- 172.21.15.52:0/596780750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff11a5bb00 con 0x55ff11a3b3d0 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292192d640 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 0x55ff11a11650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292112c640 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3aff0 0x55ff11a7c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.056+0000 7f292092b640 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a72860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f290c0011d0 tx=0x7f290c097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4fbdfaa9462143 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f29067fc640 1 -- 172.21.15.52:0/596780750 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c060710 con 0x55ff11a3b3d0 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f29067fc640 1 -- 172.21.15.52:0/596780750 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f290c092030 con 0x55ff11a3b3d0 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f29067fc640 1 -- 172.21.15.52:0/596780750 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c064ac0 con 0x55ff11a3b3d0 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f2926185740 1 -- 172.21.15.52:0/596780750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 msgr2=0x55ff11a72860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f2926185740 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a72860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f290c0011d0 tx=0x7f290c097040 comp rx=0 tx=0).stop 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f2926185740 1 -- 172.21.15.52:0/596780750 shutdown_connections 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f2926185740 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a54570 0x55ff11a11650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f2926185740 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a72860 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f2926185740 1 --2- 172.21.15.52:0/596780750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a3aff0 0x55ff11a7c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f2926185740 1 -- 172.21.15.52:0/596780750 >> 172.21.15.52:0/596780750 conn(0x55ff11a0dc50 msgr2=0x55ff11a40540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f2926185740 1 -- 172.21.15.52:0/596780750 shutdown_connections 2024-05-11T01:01:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.057+0000 7f2926185740 1 -- 172.21.15.52:0/596780750 wait complete. 2024-05-11T01:01:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.058+0000 7f2926185740 1 -- start start 2024-05-11T01:01:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.058+0000 7f2926185740 1 Processor -- start 2024-05-11T01:01:51.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f2926185740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a3aff0 0x55ff11a7c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f2926185740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f2926185740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a54570 0x55ff11a3ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f2926185740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ff11a41650 con 0x55ff11a3b3d0 2024-05-11T01:01:51.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f2926185740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ff11a13070 con 0x55ff11a54570 2024-05-11T01:01:51.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f2926185740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x55ff11a549f0 con 0x55ff11a3aff0 2024-05-11T01:01:51.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f2926185740 10 client.0 _ll_register_callbacks cb 0x7f290c001120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292092b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292092b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40922/0 (socket says 172.21.15.52:40922) 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292092b640 1 -- 172.21.15.52:0/799578891 learned_addr learned my addr 172.21.15.52:0/799578891 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292192d640 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a54570 0x55ff11a3ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292112c640 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a3aff0 0x55ff11a7c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292092b640 1 -- 172.21.15.52:0/799578891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a3aff0 msgr2=0x55ff11a7c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292092b640 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a3aff0 0x55ff11a7c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292092b640 1 -- 172.21.15.52:0/799578891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a54570 msgr2=0x55ff11a3ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292092b640 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a54570 0x55ff11a3ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292092b640 1 -- 172.21.15.52:0/799578891 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff11a59200 con 0x55ff11a3b3d0 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f292092b640 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3e810 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f290c0601d0 tx=0x7f2918038040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecf4b63bb81c13a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f28fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-11T01:01:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f28fbfff640 1 -- 172.21.15.52:0/799578891 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c060710 con 0x55ff11a3b3d0 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f28fbfff640 1 -- 172.21.15.52:0/799578891 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f290c075b60 con 0x55ff11a3b3d0 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.059+0000 7f2926185740 1 -- 172.21.15.52:0/799578891 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c092030 con 0x55ff11a3b3d0 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f28fbfff640 1 -- 172.21.15.52:0/799578891 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f290c05f970 con 0x55ff11a3b3d0 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f2926185740 10 client.5019 fetch_fsmap learned FSMap version 17 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f2926185740 1 -- 172.21.15.52:0/799578891 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55ff11a517e0 con 0x55ff11a3b3d0 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f28fbfff640 1 -- 172.21.15.52:0/799578891 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f290c092030 con 0x55ff11a3b3d0 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f28fbfff640 1 -- 172.21.15.52:0/799578891 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f290c05fb10 con 0x55ff11a3b3d0 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f2926185740 10 client.5019 fetch_fsmap finished waiting for FSMap version 17 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f2926185740 10 client.5019 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f2926185740 20 client.5019 populate_metadata read hostname 'smithi052' 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f2926185740 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] conn(0x55ff11a6add0 0x55ff11a56f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f2926185740 4 client.5019 mds_command: new command op to 4854 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f2926185740 1 -- 172.21.15.52:0/799578891 --> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] -- command(tid 0: {"prefix": "status"}) -- 0x55ff119854e0 con 0x55ff11a6add0 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f292112c640 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] conn(0x55ff11a6add0 0x55ff11a56f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f28fbfff640 10 client.5019 ms_handle_connect on v2:172.21.15.52:6832/1325827549 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.060+0000 7f292112c640 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] conn(0x55ff11a6add0 0x55ff11a56f30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b37cd6e2c42b414 server_cookie=6a4c9150687fe80d in_seq=0 out_seq=0 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f28fbfff640 1 -- 172.21.15.52:0/799578891 <== mds.0 v2:172.21.15.52:6832/1325827549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ff119854e0 con 0x55ff11a6add0 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f28fbfff640 10 client.5019 handle_command_reply: tid=0 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 client.5019 shutdown 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 -- 172.21.15.52:0/799578891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 msgr2=0x55ff11a3e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3e810 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f290c0601d0 tx=0x7f2918038040 comp rx=0 tx=0).stop 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 -- 172.21.15.52:0/799578891 shutdown_connections 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.52:6832/1325827549,v1:172.21.15.52:6834/1325827549] conn(0x55ff11a6add0 0x55ff11a56f30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ff11a54570 0x55ff11a3ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff11a3b3d0 0x55ff11a3e810 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f28fbfff640 0 client.5019 ms_handle_reset on v2:172.21.15.52:6832/1325827549 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 --2- 172.21.15.52:0/799578891 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55ff11a3aff0 0x55ff11a7c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 -- 172.21.15.52:0/799578891 >> 172.21.15.52:0/799578891 conn(0x55ff11a0dc50 msgr2=0x55ff11a54b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 -- 172.21.15.52:0/799578891 shutdown_connections 2024-05-11T01:01:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 1 -- 172.21.15.52:0/799578891 wait complete. 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2926185740 20 client.5019 trim_cache size 0 max 16384 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "f8411cb1-0508-49cc-906f-2364b6835afe", 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 4854, 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 5.7906954229999998, 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 17, 2024-05-11T01:01:51.069 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 31, 2024-05-11T01:01:51.070 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 31, 2024-05-11T01:01:51.070 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 12.659767036 2024-05-11T01:01:51.070 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:01:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2923390640 1 -- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29000762d0 msgr2=0x7f2900078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2923390640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29000762d0 0x7f2900078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2918033000 tx=0x7f2918007000 comp rx=0 tx=0).stop 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2923390640 1 -- 172.21.15.52:0/2196099038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f291c14ddf0 msgr2=0x7f291c14e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.061+0000 7f2923390640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f291c14ddf0 0x7f291c14e250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f290c00f820 tx=0x7f290c00fce0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.062+0000 7f2923390640 1 -- 172.21.15.52:0/2196099038 shutdown_connections 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.062+0000 7f2923390640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f29000762d0 0x7f2900078790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.062+0000 7f2923390640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f291c0cd260 0x7f291c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.062+0000 7f2923390640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f291c14ddf0 0x7f291c14e250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.062+0000 7f2923390640 1 --2- 172.21.15.52:0/2196099038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2908003a30 0x7f291c0d3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.062+0000 7f2923390640 1 -- 172.21.15.52:0/2196099038 >> 172.21.15.52:0/2196099038 conn(0x7f291c08eb70 msgr2=0x7f291c0b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.062+0000 7f2923390640 1 -- 172.21.15.52:0/2196099038 shutdown_connections 2024-05-11T01:01:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.062+0000 7f2923390640 1 -- 172.21.15.52:0/2196099038 wait complete. 2024-05-11T01:01:51.072 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T01:01:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.176+0000 7fb1a0d33640 1 Processor -- start 2024-05-11T01:01:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.176+0000 7fb1a0d33640 1 -- start start 2024-05-11T01:01:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.176+0000 7fb1a0d33640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb19c0d3eb0 0x7fb19c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb1a0d33640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb19c0d15b0 0x7fb19c0d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb1a0d33640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb19c0d0510 con 0x7fb19c15f530 2024-05-11T01:01:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb1a0d33640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb19c0d5d00 con 0x7fb19c0d3eb0 2024-05-11T01:01:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb1a0d33640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb19c0d07f0 con 0x7fb19c0d15b0 2024-05-11T01:01:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb199d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb19c0d3eb0 0x7fb19c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb199d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb19c0d3eb0 0x7fb19c0cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40924/0 (socket says 172.21.15.52:40924) 2024-05-11T01:01:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb199d9b640 1 -- 172.21.15.52:0/1359953158 learned_addr learned my addr 172.21.15.52:0/1359953158 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb19ad9d640 1 --2- 172.21.15.52:0/1359953158 >> v2:172.21.15.119:3300/0 conn(0x7fb19c0d15b0 0x7fb19c0d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb199d9b640 1 -- 172.21.15.52:0/1359953158 >> v2:172.21.15.119:3300/0 conn(0x7fb19c0d15b0 msgr2=0x7fb19c0d57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb199d9b640 1 --2- 172.21.15.52:0/1359953158 >> v2:172.21.15.119:3300/0 conn(0x7fb19c0d15b0 0x7fb19c0d57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb199d9b640 1 -- 172.21.15.52:0/1359953158 >> v1:172.21.15.119:6789/0 conn(0x7fb19c15f530 legacy=0x7fb19c0d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.177+0000 7fb199d9b640 1 -- 172.21.15.52:0/1359953158 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c0d0ad0 con 0x7fb19c0d3eb0 2024-05-11T01:01:51.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.178+0000 7fb199d9b640 1 --2- 172.21.15.52:0/1359953158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c0d3eb0 0x7fb19c0cfe30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0d2d50 tx=0x7fb18402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857aac08db80686a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.178+0000 7fb19959a640 1 -- 172.21.15.52:0/1359953158 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184037070 con 0x7fb19c0d3eb0 2024-05-11T01:01:51.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.178+0000 7fb19959a640 1 -- 172.21.15.52:0/1359953158 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb184032440 con 0x7fb19c0d3eb0 2024-05-11T01:01:51.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.178+0000 7fb19959a640 1 -- 172.21.15.52:0/1359953158 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184032810 con 0x7fb19c0d3eb0 2024-05-11T01:01:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.178+0000 7fb1a0d33640 1 -- 172.21.15.52:0/1359953158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c0d3eb0 msgr2=0x7fb19c0cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.178+0000 7fb1a0d33640 1 --2- 172.21.15.52:0/1359953158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c0d3eb0 0x7fb19c0cfe30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0d2d50 tx=0x7fb18402a1e0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.179+0000 7fb1a0d33640 1 -- 172.21.15.52:0/1359953158 shutdown_connections 2024-05-11T01:01:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.179+0000 7fb1a0d33640 1 --2- 172.21.15.52:0/1359953158 >> v2:172.21.15.119:3300/0 conn(0x7fb19c0d15b0 0x7fb19c0d57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.179+0000 7fb1a0d33640 1 --2- 172.21.15.52:0/1359953158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c0d3eb0 0x7fb19c0cfe30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.179+0000 7fb1a0d33640 1 -- 172.21.15.52:0/1359953158 >> 172.21.15.52:0/1359953158 conn(0x7fb19c08cfb0 msgr2=0x7fb19c09e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.179+0000 7fb1a0d33640 1 -- 172.21.15.52:0/1359953158 shutdown_connections 2024-05-11T01:01:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.179+0000 7fb1a0d33640 1 -- 172.21.15.52:0/1359953158 wait complete. 2024-05-11T01:01:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.180+0000 7fb1a0d33640 1 Processor -- start 2024-05-11T01:01:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.180+0000 7fb1a0d33640 1 -- start start 2024-05-11T01:01:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb1a0d33640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb19c0d15b0 0x7fb19c0b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb1a0d33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb19c0d3eb0 0x7fb19c158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb1a0d33640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c15f530 0x7fb19c0b9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb1a0d33640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c15f910 con 0x7fb19c15f530 2024-05-11T01:01:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb1a0d33640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c0d1090 con 0x7fb19c0d3eb0 2024-05-11T01:01:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb1a0d33640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb19c15fa90 con 0x7fb19c0d15b0 2024-05-11T01:01:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c15f530 0x7fb19c0b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c15f530 0x7fb19c0b9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40930/0 (socket says 172.21.15.52:40930) 2024-05-11T01:01:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19ad9d640 1 -- 172.21.15.52:0/576050935 learned_addr learned my addr 172.21.15.52:0/576050935 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb199d9b640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb19c0d3eb0 0x7fb19c158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19a59c640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb19c0d15b0 0x7fb19c0b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19ad9d640 1 -- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb19c0d15b0 msgr2=0x7fb19c0b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19ad9d640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb19c0d15b0 0x7fb19c0b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19ad9d640 1 -- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb19c0d3eb0 msgr2=0x7fb19c158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19ad9d640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb19c0d3eb0 0x7fb19c158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19ad9d640 1 -- 172.21.15.52:0/576050935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c0b8690 con 0x7fb19c15f530 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.181+0000 7fb19ad9d640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c15f530 0x7fb19c0b9730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb19400d800 tx=0x7fb19400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.182+0000 7fb192ffd640 1 -- 172.21.15.52:0/576050935 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19400b6c0 con 0x7fb19c15f530 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.182+0000 7fb192ffd640 1 -- 172.21.15.52:0/576050935 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb194019070 con 0x7fb19c15f530 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.182+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19c0b8970 con 0x7fb19c15f530 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.182+0000 7fb192ffd640 1 -- 172.21.15.52:0/576050935 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19400bac0 con 0x7fb19c15f530 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.182+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19c0b8f30 con 0x7fb19c15f530 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.183+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19c0696a0 con 0x7fb19c15f530 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.183+0000 7fb192ffd640 1 -- 172.21.15.52:0/576050935 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb19400bc60 con 0x7fb19c15f530 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.184+0000 7fb192ffd640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb17c0762d0 0x7fb17c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.184+0000 7fb192ffd640 1 -- 172.21.15.52:0/576050935 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fb1940979c0 con 0x7fb19c15f530 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.184+0000 7fb19a59c640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb17c0762d0 0x7fb17c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:51.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.185+0000 7fb19a59c640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb17c0762d0 0x7fb17c078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b450 tx=0x7fb18c00d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.189+0000 7fb192ffd640 1 -- 172.21.15.52:0/576050935 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb194060c30 con 0x7fb19c15f530 2024-05-11T01:01:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.419+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb19c0d5940 con 0x7fb19c15f530 2024-05-11T01:01:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.419+0000 7fb192ffd640 1 -- 172.21.15.52:0/576050935 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fb194064f30 con 0x7fb19c15f530 2024-05-11T01:01:51.420 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T01:01:51.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb17c0762d0 msgr2=0x7fb17c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb17c0762d0 0x7fb17c078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b450 tx=0x7fb18c00d040 comp rx=0 tx=0).stop 2024-05-11T01:01:51.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c15f530 msgr2=0x7fb19c0b9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:51.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c15f530 0x7fb19c0b9730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb19400d800 tx=0x7fb19400dcc0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 shutdown_connections 2024-05-11T01:01:51.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb17c0762d0 0x7fb17c078790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb19c15f530 0x7fb19c0b9730 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb19c0d3eb0 0x7fb19c158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 --2- 172.21.15.52:0/576050935 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb19c0d15b0 0x7fb19c0b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:51.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 >> 172.21.15.52:0/576050935 conn(0x7fb19c08cfb0 msgr2=0x7fb19c0d2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:51.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 shutdown_connections 2024-05-11T01:01:51.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51.421+0000 7fb1a0d33640 1 -- 172.21.15.52:0/576050935 wait complete. 2024-05-11T01:01:51.433 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T01:01:51.433 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T01:01:51.433 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-05-11T01:01:51.433 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T01:01:51.433 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:51.434 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T01:01:51.434 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T01:01:51.434 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T01:01:51.434 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T01:01:51.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:51.435 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-05-11T01:01:51.452 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:01:51.452 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:01:51.452 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:01:51.452 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-11T01:01:51.453 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42730sec preferred_lft 42730sec 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 1e:e6:90:cd:c4:f4 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::8c71:f4ff:fe68:49bf/64 scope link 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:51.454 INFO:teuthology.orchestra.run.smithi052.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T01:01:51.455 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 1e:e6:90:cd:c4:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:51.455 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::1ce6:90ff:fecd:c4f4/64 scope link 2024-05-11T01:01:51.455 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:51.455 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T01:01:51.455 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 72:00:c1:6e:a1:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:51.455 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::7000:c1ff:fe6e:a1e0/64 scope link 2024-05-11T01:01:51.455 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:01:51.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:51.455 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T01:01:51.517 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-05-11T01:01:51.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:51.520 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-05-11T01:01:51.581 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-05-11T01:01:51.581 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T01:01:51.582 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-11T01:01:51.582 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-11T01:01:51.582 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T01:01:51.582 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:01:51.582 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:01:51.582 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-11T01:01:51.582 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-11T01:01:51.582 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-11T01:01:51.582 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:01:51.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:01:51.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:51.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:51.712 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-11T01:01:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T01:01:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T01:01:51.728 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T01:01:51.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:51.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-05-11T01:01:51.796 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-11T01:01:51.796 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:51.796 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:51.853 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T01:01:51.853 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:01:51.853 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:51.909 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-11T01:01:51.975 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-11T01:01:52.033 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-11T01:01:52.034 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T01:01:52.035 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T01:01:52.035 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T01:01:52.035 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T01:01:52.035 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-11T01:01:52.035 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:52.035 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:52.035 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:52.035 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:52.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:52.035 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-11T01:01:52.089 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-11T01:01:52.089 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-11T01:01:52.131 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-11T01:01:52.158 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:01:52.215 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-11T01:01:52.215 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-11T01:01:52.215 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-11T01:01:52.215 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-11T01:01:52.215 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-11T01:01:52.215 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-11T01:01:52.215 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-11T01:01:52.215 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-11T01:01:52.215 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:01:52.216 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:01:52.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:52.218 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-11T01:01:52.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:2024-05-11T01:01:52.216+0000 7fcff1722480 -1 init, newargv = 0x56083588bc40 newargc=15 2024-05-11T01:01:52.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[47063]: starting ceph client 2024-05-11T01:01:52.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[47063]: starting fuse 2024-05-11T01:01:52.235 INFO:teuthology.orchestra.run.smithi052.stdout:40 2024-05-11T01:01:52.235 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-11T01:01:52.235 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-05-11T01:01:52.236 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-05-11T01:01:52.237 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-05-11T01:01:52.237 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:52.237 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-05-11T01:01:52.237 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T01:01:52.237 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T01:01:52.238 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T01:01:52.238 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-05-11T01:01:52.238 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-05-11T01:01:52.238 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-05-11T01:01:52.238 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T01:01:52.238 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:01:52.238 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.0")) 2024-05-11T01:01:52.238 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-05-11T01:01:52.359 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.0.47063.asok 2024-05-11T01:01:52.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:01:52.369 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47063.asok 2024-05-11T01:01:52.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:52.369 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47063.asok status 2024-05-11T01:01:52.508 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "10380055a1fc5e7d3b37b252af7045f5c2c8ba29", 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-3570-g10380055 (10380055a1fc5e7d3b37b252af7045f5c2c8ba29) squid (dev)", 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "0", 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "47063", 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 5034, 2024-05-11T01:01:52.509 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 5034 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2069599575 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-11T01:01:52.510 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2069599575 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.5034 192.168.0.1:0/2069599575", 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/2069599575", 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 17, 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 31, 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-05-11T01:01:52.511 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:01:52.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:52.517 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-05-11T01:01:52.517 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:52.534 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-05-11T01:01:52.534 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:52.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:52.534 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T01:01:52.601 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-11T01:01:52.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.746+0000 7fe78deda640 1 Processor -- start 2024-05-11T01:01:52.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.746+0000 7fe78deda640 1 -- start start 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.746+0000 7fe78deda640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe78815f430 0x7fe78815fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.746+0000 7fe78deda640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.746+0000 7fe78deda640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe788163250 con 0x7fe7881602b0 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.746+0000 7fe78deda640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fe788163530 con 0x7fe78815f430 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.746+0000 7fe78deda640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe788163810 con 0x7fe7880d3d90 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe787fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe7877fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe78815f430 0x7fe78815fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe7877fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe78815f430 0x7fe78815fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:54510/0 (socket says 172.21.15.52:54510) 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe7877fe640 1 -- 172.21.15.52:0/2658176779 learned_addr learned my addr 172.21.15.52:0/2658176779 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe787fff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880d2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:41400/0 (socket says 172.21.15.52:41400) 2024-05-11T01:01:52.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe786ffd640 1 --1- 172.21.15.52:0/2658176779 >> v1:172.21.15.119:6789/0 conn(0x7fe7881602b0 0x7fe7880d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:48174/0 (socket says 172.21.15.52:48174) 2024-05-11T01:01:52.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe787fff640 1 -- 172.21.15.52:0/2658176779 >> v2:172.21.15.119:3300/0 conn(0x7fe78815f430 msgr2=0x7fe78815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:52.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe787fff640 1 --2- 172.21.15.52:0/2658176779 >> v2:172.21.15.119:3300/0 conn(0x7fe78815f430 0x7fe78815fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:52.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe787fff640 1 -- 172.21.15.52:0/2658176779 >> v1:172.21.15.119:6789/0 conn(0x7fe7881602b0 legacy=0x7fe7880d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:52.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe787fff640 1 -- 172.21.15.52:0/2658176779 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe788163af0 con 0x7fe7880d3d90 2024-05-11T01:01:52.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.747+0000 7fe7877fe640 1 --2- 172.21.15.52:0/2658176779 >> v2:172.21.15.119:3300/0 conn(0x7fe78815f430 0x7fe78815fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:52.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.748+0000 7fe787fff640 1 --2- 172.21.15.52:0/2658176779 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880d2260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe780009d90 tx=0x7fe78002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1497faf3b992ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:52.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.748+0000 7fe7867fc640 1 -- 172.21.15.52:0/2658176779 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780007740 con 0x7fe7880d3d90 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.748+0000 7fe7867fc640 1 -- 172.21.15.52:0/2658176779 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7800078e0 con 0x7fe7880d3d90 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.748+0000 7fe7867fc640 1 -- 172.21.15.52:0/2658176779 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780007cb0 con 0x7fe7880d3d90 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.749+0000 7fe78deda640 1 -- 172.21.15.52:0/2658176779 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 msgr2=0x7fe7880d2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.749+0000 7fe78deda640 1 --2- 172.21.15.52:0/2658176779 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880d2260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe780009d90 tx=0x7fe78002a660 comp rx=0 tx=0).stop 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.749+0000 7fe78deda640 1 -- 172.21.15.52:0/2658176779 shutdown_connections 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.749+0000 7fe78deda640 1 --2- 172.21.15.52:0/2658176779 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880d2260 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.749+0000 7fe78deda640 1 --2- 172.21.15.52:0/2658176779 >> v2:172.21.15.119:3300/0 conn(0x7fe78815f430 0x7fe78815fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.749+0000 7fe78deda640 1 -- 172.21.15.52:0/2658176779 >> 172.21.15.52:0/2658176779 conn(0x7fe78808eb70 msgr2=0x7fe78809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.749+0000 7fe78deda640 1 -- 172.21.15.52:0/2658176779 shutdown_connections 2024-05-11T01:01:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.749+0000 7fe78deda640 1 -- 172.21.15.52:0/2658176779 wait complete. 2024-05-11T01:01:52.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.750+0000 7fe78deda640 1 Processor -- start 2024-05-11T01:01:52.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.750+0000 7fe78deda640 1 -- start start 2024-05-11T01:01:52.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe78deda640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:52.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe78deda640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe78815f430 0x7fe7880cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe78deda640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7881602b0 0x7fe788161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe78deda640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe788163030 con 0x7fe78815f430 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe78deda640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe788162eb0 con 0x7fe7881602b0 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe78deda640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe7880d27a0 con 0x7fe7880d3d90 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe786ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe78815f430 0x7fe7880cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe786ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe78815f430 0x7fe7880cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47462/0 (socket says 172.21.15.52:47462) 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe786ffd640 1 -- 172.21.15.52:0/603087865 learned_addr learned my addr 172.21.15.52:0/603087865 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe787fff640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7881602b0 0x7fe788161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.751+0000 7fe7877fe640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe786ffd640 1 -- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 msgr2=0x7fe7880ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe786ffd640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe786ffd640 1 -- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7881602b0 msgr2=0x7fe788161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe786ffd640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7881602b0 0x7fe788161fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe786ffd640 1 -- 172.21.15.52:0/603087865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe788160f30 con 0x7fe78815f430 2024-05-11T01:01:52.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe786ffd640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe78815f430 0x7fe7880cb410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe77c001eb0 tx=0x7fe77c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:52.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe76ffff640 1 -- 172.21.15.52:0/603087865 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c0074e0 con 0x7fe78815f430 2024-05-11T01:01:52.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe76ffff640 1 -- 172.21.15.52:0/603087865 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe77c007680 con 0x7fe78815f430 2024-05-11T01:01:52.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe76ffff640 1 -- 172.21.15.52:0/603087865 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c0049e0 con 0x7fe78815f430 2024-05-11T01:01:52.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe788161210 con 0x7fe78815f430 2024-05-11T01:01:52.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.752+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7881617d0 con 0x7fe78815f430 2024-05-11T01:01:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.753+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe78806a040 con 0x7fe78815f430 2024-05-11T01:01:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.754+0000 7fe76ffff640 1 -- 172.21.15.52:0/603087865 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe77c01a020 con 0x7fe78815f430 2024-05-11T01:01:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.754+0000 7fe76ffff640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe7740762d0 0x7fe774078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.755+0000 7fe76ffff640 1 -- 172.21.15.52:0/603087865 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fe77c096c70 con 0x7fe78815f430 2024-05-11T01:01:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.755+0000 7fe7877fe640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe7740762d0 0x7fe774078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:52.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.755+0000 7fe7877fe640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe7740762d0 0x7fe774078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe778006ca0 tx=0x7fe778009240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:52.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.759+0000 7fe76ffff640 1 -- 172.21.15.52:0/603087865 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe78806a040 con 0x7fe78815f430 2024-05-11T01:01:52.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.978+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe7880baea0 con 0x7fe7740762d0 2024-05-11T01:01:52.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.980+0000 7fe76ffff640 1 -- 172.21.15.52:0/603087865 <== mgr.4102 v2:172.21.15.119:6832/3083030853 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe7880baea0 con 0x7fe7740762d0 2024-05-11T01:01:52.980 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-05-11T01:01:52.980 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:01:52.980 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephfs" 2024-05-11T01:01:52.980 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:01:52.980 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe7740762d0 msgr2=0x7fe774078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe7740762d0 0x7fe774078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe778006ca0 tx=0x7fe778009240 comp rx=0 tx=0).stop 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe78815f430 msgr2=0x7fe7880cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe78815f430 0x7fe7880cb410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe77c001eb0 tx=0x7fe77c00c890 comp rx=0 tx=0).stop 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 shutdown_connections 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fe7740762d0 0x7fe774078790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7881602b0 0x7fe788161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe78815f430 0x7fe7880cb410 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 --2- 172.21.15.52:0/603087865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe7880d3d90 0x7fe7880ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 >> 172.21.15.52:0/603087865 conn(0x7fe78808eb70 msgr2=0x7fe78815e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 shutdown_connections 2024-05-11T01:01:52.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:52.982+0000 7fe78deda640 1 -- 172.21.15.52:0/603087865 wait complete. 2024-05-11T01:01:52.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-05-11T01:01:53.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.099+0000 7fbe0402a640 1 Processor -- start 2024-05-11T01:01:53.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.099+0000 7fbe0402a640 1 -- start start 2024-05-11T01:01:53.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe0402a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbdfc15f430 0x7fbdfc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:53.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe0402a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbdfc0b5420 0x7fbdfc0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:53.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe0402a640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdfc0cfc60 con 0x7fbdfc0d1eb0 2024-05-11T01:01:53.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe0402a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fbdfc0d2a80 con 0x7fbdfc15f430 2024-05-11T01:01:53.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe0402a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbdfc163120 con 0x7fbdfc0b5420 2024-05-11T01:01:53.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe01dc6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbdfc15f430 0x7fbdfc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:53.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe025c7640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbdfc0d1eb0 0x7fbdfc0d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51132/0 (socket says 172.21.15.52:51132) 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe01dc6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fbdfc15f430 0x7fbdfc0b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47476/0 (socket says 172.21.15.52:47476) 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe025c7640 1 -- 172.21.15.52:0/2542577572 learned_addr learned my addr 172.21.15.52:0/2542577572 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe015c5640 1 --2- 172.21.15.52:0/2542577572 >> v2:172.21.15.119:3300/0 conn(0x7fbdfc0b5420 0x7fbdfc0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe01dc6640 1 -- 172.21.15.52:0/2542577572 >> v2:172.21.15.119:3300/0 conn(0x7fbdfc0b5420 msgr2=0x7fbdfc0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe01dc6640 1 --2- 172.21.15.52:0/2542577572 >> v2:172.21.15.119:3300/0 conn(0x7fbdfc0b5420 0x7fbdfc0d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe01dc6640 1 -- 172.21.15.52:0/2542577572 >> v1:172.21.15.52:6789/0 conn(0x7fbdfc0d1eb0 legacy=0x7fbdfc0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe00dc4640 1 -- 172.21.15.52:0/2542577572 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3067026792 0 0) 0x7fbdfc0cfc60 con 0x7fbdfc0d1eb0 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.100+0000 7fbe01dc6640 1 -- 172.21.15.52:0/2542577572 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc0d24c0 con 0x7fbdfc15f430 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.101+0000 7fbe01dc6640 1 --2- 172.21.15.52:0/2542577572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc15f430 0x7fbdfc0b4ee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc08d7f0 tx=0x7fbdf802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73700df43589e4d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.101+0000 7fbe00dc4640 1 -- 172.21.15.52:0/2542577572 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8037070 con 0x7fbdfc15f430 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.101+0000 7fbe00dc4640 1 -- 172.21.15.52:0/2542577572 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf8032440 con 0x7fbdfc15f430 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.101+0000 7fbe00dc4640 1 -- 172.21.15.52:0/2542577572 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8032810 con 0x7fbdfc15f430 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.102+0000 7fbe0402a640 1 -- 172.21.15.52:0/2542577572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc15f430 msgr2=0x7fbdfc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.102+0000 7fbe0402a640 1 --2- 172.21.15.52:0/2542577572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc15f430 0x7fbdfc0b4ee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc08d7f0 tx=0x7fbdf802a1a0 comp rx=0 tx=0).stop 2024-05-11T01:01:53.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.102+0000 7fbe0402a640 1 -- 172.21.15.52:0/2542577572 shutdown_connections 2024-05-11T01:01:53.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.102+0000 7fbe0402a640 1 --2- 172.21.15.52:0/2542577572 >> v2:172.21.15.119:3300/0 conn(0x7fbdfc0b5420 0x7fbdfc0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:53.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.102+0000 7fbe0402a640 1 --2- 172.21.15.52:0/2542577572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc15f430 0x7fbdfc0b4ee0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:53.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.102+0000 7fbe0402a640 1 -- 172.21.15.52:0/2542577572 >> 172.21.15.52:0/2542577572 conn(0x7fbdfc08ebb0 msgr2=0x7fbdfc09e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:53.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.102+0000 7fbe0402a640 1 -- 172.21.15.52:0/2542577572 shutdown_connections 2024-05-11T01:01:53.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.102+0000 7fbe0402a640 1 -- 172.21.15.52:0/2542577572 wait complete. 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.103+0000 7fbe0402a640 1 Processor -- start 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.103+0000 7fbe0402a640 1 -- start start 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe0402a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc0b5420 0x7fbdfc0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe0402a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbdfc0d1eb0 0x7fbdfc15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe0402a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdfc15f430 0x7fbdfc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe0402a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc0d01e0 con 0x7fbdfc0b5420 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe0402a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc0cff00 con 0x7fbdfc15f430 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe0402a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbdfc0d0e10 con 0x7fbdfc0d1eb0 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe01dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc0b5420 0x7fbdfc0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe01dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc0b5420 0x7fbdfc0ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47482/0 (socket says 172.21.15.52:47482) 2024-05-11T01:01:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe01dc6640 1 -- 172.21.15.52:0/349707964 learned_addr learned my addr 172.21.15.52:0/349707964 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe01dc6640 1 -- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbdfc0d1eb0 msgr2=0x7fbdfc15b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe01dc6640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbdfc0d1eb0 0x7fbdfc15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe01dc6640 1 -- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdfc15f430 msgr2=0x7fbdfc06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe01dc6640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdfc15f430 0x7fbdfc06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe01dc6640 1 -- 172.21.15.52:0/349707964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc15bf20 con 0x7fbdfc0b5420 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe01dc6640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc0b5420 0x7fbdfc0ce140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8000c00 tx=0x7fbdf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbdf27fc640 1 -- 172.21.15.52:0/349707964 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8037040 con 0x7fbdfc0b5420 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbdf27fc640 1 -- 172.21.15.52:0/349707964 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf8042070 con 0x7fbdfc0b5420 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.104+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdfc15c200 con 0x7fbdfc0b5420 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.105+0000 7fbdf27fc640 1 -- 172.21.15.52:0/349707964 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8030a70 con 0x7fbdfc0b5420 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.105+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdfc15c7c0 con 0x7fbdfc0b5420 2024-05-11T01:01:53.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.106+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdfc0695d0 con 0x7fbdfc0b5420 2024-05-11T01:01:53.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.106+0000 7fbdf27fc640 1 -- 172.21.15.52:0/349707964 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbdf8030c10 con 0x7fbdfc0b5420 2024-05-11T01:01:53.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.107+0000 7fbdf27fc640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbde0076300 0x7fbde00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:53.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.107+0000 7fbdf27fc640 1 -- 172.21.15.52:0/349707964 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fbdf80b7ff0 con 0x7fbdfc0b5420 2024-05-11T01:01:53.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.107+0000 7fbe015c5640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbde0076300 0x7fbde00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:53.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.108+0000 7fbe015c5640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbde0076300 0x7fbde00787c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbdec006fd0 tx=0x7fbdec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:53.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.111+0000 7fbdf27fc640 1 -- 172.21.15.52:0/349707964 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbdf80819e0 con 0x7fbdfc0b5420 2024-05-11T01:01:53.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:53.356+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fbdfc0d15b0 con 0x7fbdfc0b5420 2024-05-11T01:01:54.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.322+0000 7fbdf27fc640 1 -- 172.21.15.52:0/349707964 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) ==== 92+0+0 (secure 0 0 0) 0x7fbdf80812e0 con 0x7fbdfc0b5420 2024-05-11T01:01:54.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.326+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbde0076300 msgr2=0x7fbde00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.326+0000 7fbe0402a640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbde0076300 0x7fbde00787c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbdec006fd0 tx=0x7fbdec008040 comp rx=0 tx=0).stop 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.326+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc0b5420 msgr2=0x7fbdfc0ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbdf27fc640 1 -- 172.21.15.52:0/349707964 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 6) ==== 98218+0+0 (secure 0 0 0) 0x7fbdf807d570 con 0x7fbdfc0b5420 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc0b5420 0x7fbdfc0ce140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8000c00 tx=0x7fbdf80046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 shutdown_connections 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbde0076300 0x7fbde00787c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdfc15f430 0x7fbdfc06d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbdfc0d1eb0 0x7fbdfc15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdfc0b5420 0x7fbdfc0ce140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 >> 172.21.15.52:0/349707964 conn(0x7fbdfc08ebb0 msgr2=0x7fbdfc0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbdf27fc640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbde003b8f0 0x7fbde00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 shutdown_connections 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 --2- 172.21.15.52:0/349707964 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fbde003b8f0 0x7fbde00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.327+0000 7fbe0402a640 1 -- 172.21.15.52:0/349707964 wait complete. 2024-05-11T01:01:54.341 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-05-11T01:01:54.442 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:54.441+0000 7f8a5fe0f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T01:01:54.458 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:54.458+0000 7fb30c787040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T01:01:54.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.464+0000 7fd9f156a640 1 Processor -- start 2024-05-11T01:01:54.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.464+0000 7fd9f156a640 1 -- start start 2024-05-11T01:01:54.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9f156a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9ec15f2f0 0x7fd9ec0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9f156a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d2d60 0x7fd9ec0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9f156a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9ec0d3a90 con 0x7fd9ec0d4e40 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9f156a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ec0d3d70 con 0x7fd9ec15f2f0 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9f156a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ec0d4050 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9eaffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9ec15f2f0 0x7fd9ec0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9eaffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9ec15f2f0 0x7fd9ec0cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47512/0 (socket says 172.21.15.52:47512) 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9eaffd640 1 -- 172.21.15.52:0/823945367 learned_addr learned my addr 172.21.15.52:0/823945367 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9eb7fe640 1 --2- 172.21.15.52:0/823945367 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d2d60 0x7fd9ec0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9eaffd640 1 -- 172.21.15.52:0/823945367 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d2d60 msgr2=0x7fd9ec0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/823945367 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d2d60 0x7fd9ec0d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9eaffd640 1 -- 172.21.15.52:0/823945367 >> v1:172.21.15.119:6789/0 conn(0x7fd9ec0d4e40 legacy=0x7fd9ec0d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.466+0000 7fd9eaffd640 1 -- 172.21.15.52:0/823945367 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec0b4e60 con 0x7fd9ec15f2f0 2024-05-11T01:01:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.467+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/823945367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec15f2f0 0x7fd9ec0cfca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009870 tx=0x7fd9dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117af1bb773d2648 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.467+0000 7fd9ea7fc640 1 -- 172.21.15.52:0/823945367 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc037070 con 0x7fd9ec15f2f0 2024-05-11T01:01:54.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.467+0000 7fd9ea7fc640 1 -- 172.21.15.52:0/823945367 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc032440 con 0x7fd9ec15f2f0 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.467+0000 7fd9ea7fc640 1 -- 172.21.15.52:0/823945367 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc032810 con 0x7fd9ec15f2f0 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.468+0000 7fd9f156a640 1 -- 172.21.15.52:0/823945367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec15f2f0 msgr2=0x7fd9ec0cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.468+0000 7fd9f156a640 1 --2- 172.21.15.52:0/823945367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec15f2f0 0x7fd9ec0cfca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009870 tx=0x7fd9dc02a270 comp rx=0 tx=0).stop 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.468+0000 7fd9f156a640 1 -- 172.21.15.52:0/823945367 shutdown_connections 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.468+0000 7fd9f156a640 1 --2- 172.21.15.52:0/823945367 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d2d60 0x7fd9ec0d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.468+0000 7fd9f156a640 1 --2- 172.21.15.52:0/823945367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec15f2f0 0x7fd9ec0cfca0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.468+0000 7fd9f156a640 1 -- 172.21.15.52:0/823945367 >> 172.21.15.52:0/823945367 conn(0x7fd9ec08ebb0 msgr2=0x7fd9ec09e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.468+0000 7fd9f156a640 1 -- 172.21.15.52:0/823945367 shutdown_connections 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.468+0000 7fd9f156a640 1 -- 172.21.15.52:0/823945367 wait complete. 2024-05-11T01:01:54.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.469+0000 7fd9f156a640 1 Processor -- start 2024-05-11T01:01:54.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9f156a640 1 -- start start 2024-05-11T01:01:54.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9f156a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0d2d60 0x7fd9ec09ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9f156a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d4e40 0x7fd9ec0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9f156a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ec15f2f0 0x7fd9ec1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9f156a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec0d01e0 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9f156a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec0d3720 con 0x7fd9ec15f2f0 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9f156a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ec0d0360 con 0x7fd9ec0d4e40 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9eaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0d2d60 0x7fd9ec09ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9eaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0d2d60 0x7fd9ec09ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47522/0 (socket says 172.21.15.52:47522) 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9eaffd640 1 -- 172.21.15.52:0/3704652584 learned_addr learned my addr 172.21.15.52:0/3704652584 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9e3fff640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d4e40 0x7fd9ec0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.470+0000 7fd9eb7fe640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ec15f2f0 0x7fd9ec1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9eaffd640 1 -- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d4e40 msgr2=0x7fd9ec0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d4e40 0x7fd9ec0cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9eaffd640 1 -- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ec15f2f0 msgr2=0x7fd9ec1f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ec15f2f0 0x7fd9ec1f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9eaffd640 1 -- 172.21.15.52:0/3704652584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec160bd0 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9eaffd640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0d2d60 0x7fd9ec09ef40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000c00 tx=0x7fd9dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9e37fe640 1 -- 172.21.15.52:0/3704652584 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc041020 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9e37fe640 1 -- 172.21.15.52:0/3704652584 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc0306c0 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ec160eb0 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9e37fe640 1 -- 172.21.15.52:0/3704652584 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc037040 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.471+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ec161470 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.472+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ec0695d0 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.473+0000 7fd9e37fe640 1 -- 172.21.15.52:0/3704652584 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 6) ==== 98218+0+0 (secure 0 0 0) 0x7fd9dc044020 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.474+0000 7fd9e37fe640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9cc076320 0x7fd9cc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.474+0000 7fd9e3fff640 1 -- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9cc076320 msgr2=0x7fd9cc0787e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/3083030853 2024-05-11T01:01:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.474+0000 7fd9e37fe640 1 -- 172.21.15.52:0/3704652584 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd9dc0b7c90 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.474+0000 7fd9e3fff640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9cc076320 0x7fd9cc0787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:01:54.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.483+0000 7fd9e37fe640 1 -- 172.21.15.52:0/3704652584 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd9dc04b030 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.556 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:54.555+0000 7f8a5fe0f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:01:54.585 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:54.585+0000 7fb30c787040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:01:54.603 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:54.602+0000 7f8a5fe0f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T01:01:54.634 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:54.634+0000 7fb30c787040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T01:01:54.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.675+0000 7fd9e3fff640 1 -- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9cc076320 msgr2=0x7fd9cc0787e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/3083030853 2024-05-11T01:01:54.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.675+0000 7fd9e3fff640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9cc076320 0x7fd9cc0787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:01:54.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.677+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fd9ec0d6080 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.680+0000 7fd9e37fe640 1 -- 172.21.15.52:0/3704652584 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) ==== 138+0+0 (secure 0 0 0) 0x7fd9dc081030 con 0x7fd9ec0d2d60 2024-05-11T01:01:54.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9cc076320 msgr2=0x7fd9cc0787e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9cc076320 0x7fd9cc0787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0d2d60 msgr2=0x7fd9ec09ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0d2d60 0x7fd9ec09ef40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000c00 tx=0x7fd9dc0046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 shutdown_connections 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fd9cc076320 0x7fd9cc0787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9ec15f2f0 0x7fd9ec1f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9ec0d4e40 0x7fd9ec0cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 --2- 172.21.15.52:0/3704652584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0d2d60 0x7fd9ec09ef40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 >> 172.21.15.52:0/3704652584 conn(0x7fd9ec08ebb0 msgr2=0x7fd9ec0d24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 shutdown_connections 2024-05-11T01:01:54.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.682+0000 7fd9f156a640 1 -- 172.21.15.52:0/3704652584 wait complete. 2024-05-11T01:01:54.694 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-05-11T01:01:54.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.802+0000 7fb742189640 1 Processor -- start 2024-05-11T01:01:54.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.802+0000 7fb742189640 1 -- start start 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.803+0000 7fb742189640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb73c15f6f0 0x7fb73c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.803+0000 7fb742189640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb73c0d12d0 con 0x7fb73c0d2330 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.803+0000 7fb742189640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb73c0d1570 con 0x7fb73c0cdd60 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.803+0000 7fb742189640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb73c06dc10 con 0x7fb73c15f6f0 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.803+0000 7fb73affd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb73c0d2330 0x7fb73c0cd650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51140/0 (socket says 172.21.15.52:51140) 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.803+0000 7fb73affd640 1 -- 172.21.15.52:0/353606959 learned_addr learned my addr 172.21.15.52:0/353606959 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.803+0000 7fb73b7fe640 1 --2- 172.21.15.52:0/353606959 >> v2:172.21.15.119:3300/0 conn(0x7fb73c15f6f0 0x7fb73c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.803+0000 7fb73a7fc640 1 -- 172.21.15.52:0/353606959 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322501025 0 0) 0x7fb73c0d12d0 con 0x7fb73c0d2330 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.803+0000 7fb73a7fc640 1 -- 172.21.15.52:0/353606959 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb724003520 con 0x7fb73c0d2330 2024-05-11T01:01:54.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.804+0000 7fb73a7fc640 1 -- 172.21.15.52:0/353606959 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794788191 0 0) 0x7fb73c0d1570 con 0x7fb73c0cdd60 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.804+0000 7fb73a7fc640 1 -- 172.21.15.52:0/353606959 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb73c0d12d0 con 0x7fb73c0cdd60 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.804+0000 7fb73b7fe640 1 -- 172.21.15.52:0/353606959 >> v1:172.21.15.119:6789/0 conn(0x7fb73c0cdd60 legacy=0x7fb73c1630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.804+0000 7fb73b7fe640 1 -- 172.21.15.52:0/353606959 >> v1:172.21.15.52:6789/0 conn(0x7fb73c0d2330 legacy=0x7fb73c0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.804+0000 7fb73b7fe640 1 -- 172.21.15.52:0/353606959 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c0d1850 con 0x7fb73c15f6f0 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.804+0000 7fb73b7fe640 1 --2- 172.21.15.52:0/353606959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb73c15f6f0 0x7fb73c06d630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0097d0 tx=0x7fb72c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52c4865cfd91c961 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb73a7fc640 1 -- 172.21.15.52:0/353606959 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c037070 con 0x7fb73c15f6f0 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb73a7fc640 1 -- 172.21.15.52:0/353606959 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72c032440 con 0x7fb73c15f6f0 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb73a7fc640 1 -- 172.21.15.52:0/353606959 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c032810 con 0x7fb73c15f6f0 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb742189640 1 -- 172.21.15.52:0/353606959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb73c15f6f0 msgr2=0x7fb73c06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb742189640 1 --2- 172.21.15.52:0/353606959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb73c15f6f0 0x7fb73c06d630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0097d0 tx=0x7fb72c02a270 comp rx=0 tx=0).stop 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb742189640 1 -- 172.21.15.52:0/353606959 shutdown_connections 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb742189640 1 --2- 172.21.15.52:0/353606959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb73c15f6f0 0x7fb73c06d630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb742189640 1 -- 172.21.15.52:0/353606959 >> 172.21.15.52:0/353606959 conn(0x7fb73c0d3000 msgr2=0x7fb73c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb742189640 1 -- 172.21.15.52:0/353606959 shutdown_connections 2024-05-11T01:01:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.805+0000 7fb742189640 1 -- 172.21.15.52:0/353606959 wait complete. 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.806+0000 7fb742189640 1 Processor -- start 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.807+0000 7fb742189640 1 -- start start 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.807+0000 7fb742189640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0cdd60 0x7fb73c157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.807+0000 7fb742189640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb73c0d2330 0x7fb73c15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.807+0000 7fb742189640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb73c0b1100 0x7fb73c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.807+0000 7fb742189640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c0d3b90 con 0x7fb73c0cdd60 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.807+0000 7fb742189640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c0d1e10 con 0x7fb73c0d2330 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.807+0000 7fb742189640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb73c0ce1d0 con 0x7fb73c0b1100 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0cdd60 0x7fb73c157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0cdd60 0x7fb73c157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47536/0 (socket says 172.21.15.52:47536) 2024-05-11T01:01:54.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73b7fe640 1 -- 172.21.15.52:0/1448242061 learned_addr learned my addr 172.21.15.52:0/1448242061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73affd640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb73c0d2330 0x7fb73c15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73bfff640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb73c0b1100 0x7fb73c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73b7fe640 1 -- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb73c0b1100 msgr2=0x7fb73c158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73b7fe640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb73c0b1100 0x7fb73c158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73b7fe640 1 -- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb73c0d2330 msgr2=0x7fb73c15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73b7fe640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb73c0d2330 0x7fb73c15aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73b7fe640 1 -- 172.21.15.52:0/1448242061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c158370 con 0x7fb73c0cdd60 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73bfff640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb73c0b1100 0x7fb73c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb73b7fe640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0cdd60 0x7fb73c157d20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb72c02a2b0 tx=0x7fb72c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb723fff640 1 -- 172.21.15.52:0/1448242061 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c037040 con 0x7fb73c0cdd60 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb723fff640 1 -- 172.21.15.52:0/1448242061 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72c042070 con 0x7fb73c0cdd60 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73c0ca170 con 0x7fb73c0cdd60 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb723fff640 1 -- 172.21.15.52:0/1448242061 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c03b930 con 0x7fb73c0cdd60 2024-05-11T01:01:54.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.808+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73c0ca730 con 0x7fb73c0cdd60 2024-05-11T01:01:54.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.809+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73c0697c0 con 0x7fb73c0cdd60 2024-05-11T01:01:54.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.810+0000 7fb723fff640 1 -- 172.21.15.52:0/1448242061 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 6) ==== 98218+0+0 (secure 0 0 0) 0x7fb72c03bad0 con 0x7fb73c0cdd60 2024-05-11T01:01:54.812 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:54.808+0000 7f8a5fe0f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T01:01:54.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.810+0000 7fb723fff640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb728076320 0x7fb7280787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:54.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.811+0000 7fb73affd640 1 -- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb728076320 msgr2=0x7fb7280787e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/3083030853 2024-05-11T01:01:54.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.811+0000 7fb723fff640 1 -- 172.21.15.52:0/1448242061 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fb72c0b7ea0 con 0x7fb73c0cdd60 2024-05-11T01:01:54.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.811+0000 7fb73affd640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb728076320 0x7fb7280787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:01:54.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:54.815+0000 7fb723fff640 1 -- 172.21.15.52:0/1448242061 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb73c0697c0 con 0x7fb73c0cdd60 2024-05-11T01:01:54.852 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:54.851+0000 7f8a5fe0f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T01:01:54.857 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:54.857+0000 7fb30c787040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T01:01:54.903 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:54.904+0000 7fb30c787040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T01:01:55.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.007+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fb73c0705e0 con 0x7fb73c0cdd60 2024-05-11T01:01:55.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.010+0000 7fb723fff640 1 -- 172.21.15.52:0/1448242061 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) ==== 133+0+0 (secure 0 0 0) 0x7fb73c0705e0 con 0x7fb73c0cdd60 2024-05-11T01:01:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb73affd640 1 -- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb728076320 msgr2=0x7fb7280787e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/3083030853 2024-05-11T01:01:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb73affd640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb728076320 0x7fb7280787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:01:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb728076320 msgr2=0x7fb7280787e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:01:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb728076320 0x7fb7280787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0cdd60 msgr2=0x7fb73c157d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0cdd60 0x7fb73c157d20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb72c02a2b0 tx=0x7fb72c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 shutdown_connections 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7fb728076320 0x7fb7280787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb73c0b1100 0x7fb73c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb73c0d2330 0x7fb73c15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 --2- 172.21.15.52:0/1448242061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0cdd60 0x7fb73c157d20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 >> 172.21.15.52:0/1448242061 conn(0x7fb73c0d3000 msgr2=0x7fb73c0b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 shutdown_connections 2024-05-11T01:01:55.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.012+0000 7fb742189640 1 -- 172.21.15.52:0/1448242061 wait complete. 2024-05-11T01:01:55.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:55.023 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-05-11T01:01:55.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:55.044 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2024-05-11T01:01:55.104 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1m 2024-05-11T01:01:55.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.253+0000 7f9beaf33640 1 Processor -- start 2024-05-11T01:01:55.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.253+0000 7f9beaf33640 1 -- start start 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.253+0000 7f9beaf33640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 0x7f9be4073330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.253+0000 7f9beaf33640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9be4073870 0x7f9be40b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9beaf33640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9be40b3c90 0x7f9be4162f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9beaf33640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f9be40b40e0 con 0x7f9be40b3c90 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9beaf33640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f9be406e990 con 0x7f9be4073870 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9beaf33640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f9be415fb30 con 0x7f9be4159ab0 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be94d0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9be40b3c90 0x7f9be4162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9be4073870 0x7f9be40b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be8ccf640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 0x7f9be4073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be94d0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9be40b3c90 0x7f9be4162f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47544/0 (socket says 172.21.15.52:47544) 2024-05-11T01:01:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be94d0640 1 -- 172.21.15.52:0/1798165201 learned_addr learned my addr 172.21.15.52:0/1798165201 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be94d0640 1 -- 172.21.15.52:0/1798165201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 msgr2=0x7f9be4073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be94d0640 1 --2- 172.21.15.52:0/1798165201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 0x7f9be4073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be94d0640 1 -- 172.21.15.52:0/1798165201 >> v2:172.21.15.119:3300/0 conn(0x7f9be4073870 msgr2=0x7f9be40b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be94d0640 1 --2- 172.21.15.52:0/1798165201 >> v2:172.21.15.119:3300/0 conn(0x7f9be4073870 0x7f9be40b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be94d0640 1 -- 172.21.15.52:0/1798165201 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be415fe10 con 0x7f9be40b3c90 2024-05-11T01:01:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be3fff640 1 --2- 172.21.15.52:0/1798165201 >> v2:172.21.15.119:3300/0 conn(0x7f9be4073870 0x7f9be40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.254+0000 7f9be8ccf640 1 --2- 172.21.15.52:0/1798165201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 0x7f9be4073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.255+0000 7f9be94d0640 1 --2- 172.21.15.52:0/1798165201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be40b3c90 0x7f9be4162f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9be40cf2b0 tx=0x7f9bdc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8c576cf1516ed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.255+0000 7f9be37fe640 1 -- 172.21.15.52:0/1798165201 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc036070 con 0x7f9be40b3c90 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.255+0000 7f9be37fe640 1 -- 172.21.15.52:0/1798165201 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bdc031600 con 0x7f9be40b3c90 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.255+0000 7f9be37fe640 1 -- 172.21.15.52:0/1798165201 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0319d0 con 0x7f9be40b3c90 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.256+0000 7f9beaf33640 1 -- 172.21.15.52:0/1798165201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be40b3c90 msgr2=0x7f9be4162f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.256+0000 7f9beaf33640 1 --2- 172.21.15.52:0/1798165201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be40b3c90 0x7f9be4162f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9be40cf2b0 tx=0x7f9bdc02a6a0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.256+0000 7f9beaf33640 1 -- 172.21.15.52:0/1798165201 shutdown_connections 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.256+0000 7f9beaf33640 1 --2- 172.21.15.52:0/1798165201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be40b3c90 0x7f9be4162f30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.256+0000 7f9beaf33640 1 --2- 172.21.15.52:0/1798165201 >> v2:172.21.15.119:3300/0 conn(0x7f9be4073870 0x7f9be40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.256+0000 7f9beaf33640 1 --2- 172.21.15.52:0/1798165201 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 0x7f9be4073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.256+0000 7f9beaf33640 1 -- 172.21.15.52:0/1798165201 >> 172.21.15.52:0/1798165201 conn(0x7f9be4075ca0 msgr2=0x7f9be4076080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.256+0000 7f9beaf33640 1 -- 172.21.15.52:0/1798165201 shutdown_connections 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.256+0000 7f9beaf33640 1 -- 172.21.15.52:0/1798165201 wait complete. 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.257+0000 7f9beaf33640 1 Processor -- start 2024-05-11T01:01:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.257+0000 7f9beaf33640 1 -- start start 2024-05-11T01:01:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.258+0000 7f9beaf33640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be4073870 0x7f9be414ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.258+0000 7f9beaf33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be40b3c90 0x7f9be40d0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.258+0000 7f9beaf33640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 0x7f9be40d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.258+0000 7f9beaf33640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41606b0 con 0x7f9be4073870 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.258+0000 7f9beaf33640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41603d0 con 0x7f9be40b3c90 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.258+0000 7f9beaf33640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f9be4163470 con 0x7f9be4159ab0 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.258+0000 7f9be8ccf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be4073870 0x7f9be414ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.258+0000 7f9be8ccf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be4073870 0x7f9be414ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47558/0 (socket says 172.21.15.52:47558) 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.258+0000 7f9be8ccf640 1 -- 172.21.15.52:0/1706574597 learned_addr learned my addr 172.21.15.52:0/1706574597 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be3fff640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be40b3c90 0x7f9be40d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be94d0640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 0x7f9be40d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be8ccf640 1 -- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 msgr2=0x7f9be40d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be8ccf640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9be4159ab0 0x7f9be40d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be8ccf640 1 -- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be40b3c90 msgr2=0x7f9be40d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be8ccf640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be40b3c90 0x7f9be40d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be8ccf640 1 -- 172.21.15.52:0/1706574597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be414ef60 con 0x7f9be4073870 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be3fff640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9be40b3c90 0x7f9be40d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be8ccf640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be4073870 0x7f9be414ea20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e990 tx=0x7f9bd800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8017070 con 0x7f9be4073870 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd800c9e0 con 0x7f9be4073870 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800cdb0 con 0x7f9be4073870 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be40cb010 con 0x7f9be4073870 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.259+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be40cb5d0 con 0x7f9be4073870 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.260+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be40695d0 con 0x7f9be4073870 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.261+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 6) ==== 98218+0+0 (secure 0 0 0) 0x7f9bd801a070 con 0x7f9be4073870 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.261+0000 7f9be17fa640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 0x7f9bc80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.262+0000 7f9be3fff640 1 -- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 msgr2=0x7f9bc80787e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/3083030853 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.262+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f9bd8097400 con 0x7f9be4073870 2024-05-11T01:01:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.262+0000 7f9be3fff640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 0x7f9bc80787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:01:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.268+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9bd80607a0 con 0x7f9be4073870 2024-05-11T01:01:55.340 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:55.339+0000 7f8a5fe0f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T01:01:55.424 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:55.423+0000 7f8a5fe0f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T01:01:55.431 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:55.431+0000 7fb30c787040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T01:01:55.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.463+0000 7f9be3fff640 1 -- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 msgr2=0x7f9bc80787e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/3083030853 2024-05-11T01:01:55.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.463+0000 7f9be3fff640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 0x7f9bc80787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:01:55.465 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:55.465+0000 7f8a5fe0f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T01:01:55.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.484+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 --> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9be40ce9c0 con 0x7f9bc8076320 2024-05-11T01:01:55.520 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:55.520+0000 7fb30c787040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T01:01:55.565 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:55.565+0000 7fb30c787040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T01:01:55.628 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:55.626+0000 7f8a5fe0f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T01:01:55.671 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:55.670+0000 7f8a5fe0f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T01:01:55.719 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:55.718+0000 7f8a5fe0f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T01:01:55.735 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:55.735+0000 7fb30c787040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T01:01:55.782 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:55.782+0000 7fb30c787040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T01:01:55.830 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:55.829+0000 7f8a5fe0f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:01:55.833 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:55.834+0000 7fb30c787040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T01:01:55.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.864+0000 7f9be3fff640 1 -- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 msgr2=0x7f9bc80787e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/3083030853 2024-05-11T01:01:55.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:55.864+0000 7f9be3fff640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 0x7f9bc80787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-11T01:01:55.950 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:55.950+0000 7fb30c787040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:01:56.148 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:56.147+0000 7f8a5fe0f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T01:01:56.271 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:56.270+0000 7f8a5fe0f040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T01:01:56.286 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:56.286+0000 7fb30c787040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T01:01:56.415 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:56.415+0000 7fb30c787040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T01:01:56.481 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:56.480+0000 7f8a5fe0f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T01:01:56.530 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:56.529+0000 7f8a5fe0f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T01:01:56.636 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:56.636+0000 7fb30c787040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T01:01:56.642 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:56.641+0000 7f8a5fe0f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T01:01:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:56.665+0000 7f9be3fff640 1 -- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 msgr2=0x7f9bc80787e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/3083030853 2024-05-11T01:01:56.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:56.665+0000 7f9be3fff640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 0x7f9bc80787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-11T01:01:56.689 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:56.689+0000 7fb30c787040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T01:01:56.808 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:56.808+0000 7fb30c787040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T01:01:56.909 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:56.908+0000 7f8a5fe0f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T01:01:56.952 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:56.951+0000 7f8a5fe0f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T01:01:57.092 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:57.092+0000 7fb30c787040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T01:01:57.093 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:57.092+0000 7f8a5fe0f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T01:01:57.138 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:57.138+0000 7fb30c787040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T01:01:57.223 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:57.222+0000 7f8a5fe0f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T01:01:57.271 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:57.270+0000 7f8a5fe0f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T01:01:57.288 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:57.289+0000 7fb30c787040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T01:01:57.324 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:57.323+0000 7f8a5fe0f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T01:01:57.367 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:57.366+0000 7f8a5fe0f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T01:01:57.429 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:57.429+0000 7fb30c787040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T01:01:57.479 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:57.479+0000 7fb30c787040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T01:01:57.536 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:57.536+0000 7fb30c787040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T01:01:57.581 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:57.581+0000 7fb30c787040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T01:01:57.752 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:57.751+0000 7f8a5fe0f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T01:01:57.866 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:57.865+0000 7f8a5fe0f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T01:01:57.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:57.877+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 7) ==== 97929+0+0 (secure 0 0 0) 0x7f9bd805ca30 con 0x7f9be4073870 2024-05-11T01:01:57.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:57.878+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 msgr2=0x7f9bc80787e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:01:57.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:57.878+0000 7f9be17fa640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/3083030853,v1:172.21.15.119:6833/3083030853] conn(0x7f9bc8076320 0x7f9bc80787e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:58.011 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.008+0000 7f89ddb66640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.011 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.008+0000 7f89ddb66640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.011 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.008+0000 7f89ddb66640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.011 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.008+0000 7f89ddb66640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.011 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.008+0000 7f89ddb66640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.011 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.009+0000 7f89dab60640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.011 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.009+0000 7f89dab60640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.011 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.009+0000 7f89dab60640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.011 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.009+0000 7f89dab60640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.012 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:01:58.009+0000 7f89dab60640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:01:58.012 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:58.012+0000 7fb30c787040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T01:01:58.135 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:01:58.135+0000 7fb30c787040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T01:01:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:58.880+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 8) ==== 98056+0+0 (secure 0 0 0) 0x7f9bd8060a00 con 0x7f9be4073870 2024-05-11T01:01:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:58.880+0000 7f9be17fa640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9bc807e1d0 0x7f9bc80805c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:58.880+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9bd8068e10 con 0x7f9bc807e1d0 2024-05-11T01:01:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:58.881+0000 7f9be3fff640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9bc807e1d0 0x7f9bc80805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:58.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:58.887+0000 7f9be3fff640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9bc807e1d0 0x7f9bc80805c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9be4151f20 tx=0x7f9bd0009190 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:59.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.044+0000 7f9be17fa640 1 -- 172.21.15.52:0/1706574597 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9bd8068e10 con 0x7f9bc807e1d0 2024-05-11T01:01:59.044 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir0 2024-05-11T01:01:59.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.046+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9bc807e1d0 msgr2=0x7f9bc80805c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.046+0000 7f9beaf33640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9bc807e1d0 0x7f9bc80805c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9be4151f20 tx=0x7f9bd0009190 comp rx=0 tx=0).stop 2024-05-11T01:01:59.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.046+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be4073870 msgr2=0x7f9be414ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.046+0000 7f9beaf33640 1 --2- 172.21.15.52:0/1706574597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9be4073870 0x7f9be414ea20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e990 tx=0x7f9bd800ee50 comp rx=0 tx=0).stop 2024-05-11T01:01:59.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.046+0000 7f9be94d0640 1 -- 172.21.15.52:0/1706574597 reap_dead start 2024-05-11T01:01:59.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.047+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 shutdown_connections 2024-05-11T01:01:59.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.047+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 >> 172.21.15.52:0/1706574597 conn(0x7f9be4075ca0 msgr2=0x7f9be40681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:59.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.047+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 shutdown_connections 2024-05-11T01:01:59.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.047+0000 7f9beaf33640 1 -- 172.21.15.52:0/1706574597 wait complete. 2024-05-11T01:01:59.059 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2024-05-11T01:01:59.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:59.060 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2024-05-11T01:01:59.079 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1m 2024-05-11T01:01:59.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.228+0000 7f5899a6d640 1 Processor -- start 2024-05-11T01:01:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.228+0000 7f5899a6d640 1 -- start start 2024-05-11T01:01:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.229+0000 7f5899a6d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f589415c0d0 0x7f5894072a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.229+0000 7f5899a6d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f589406e880 0x7f589415fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.229+0000 7f5899a6d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f589415c4b0 con 0x7f589406c480 2024-05-11T01:01:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.229+0000 7f5899a6d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5894160ad0 con 0x7f589415c0d0 2024-05-11T01:01:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.229+0000 7f5899a6d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5894160510 con 0x7f589406e880 2024-05-11T01:01:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.229+0000 7f58937fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f589415c0d0 0x7f5894072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.229+0000 7f58937fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f589415c0d0 0x7f5894072a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47572/0 (socket says 172.21.15.52:47572) 2024-05-11T01:01:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.229+0000 7f58937fe640 1 -- 172.21.15.52:0/3024520318 learned_addr learned my addr 172.21.15.52:0/3024520318 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.229+0000 7f588affd640 1 --2- 172.21.15.52:0/3024520318 >> v2:172.21.15.119:3300/0 conn(0x7f589406e880 0x7f589415fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.230+0000 7f58937fe640 1 -- 172.21.15.52:0/3024520318 >> v2:172.21.15.119:3300/0 conn(0x7f589406e880 msgr2=0x7f589415fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:01:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.230+0000 7f58937fe640 1 --2- 172.21.15.52:0/3024520318 >> v2:172.21.15.119:3300/0 conn(0x7f589406e880 0x7f589415fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.230+0000 7f58937fe640 1 -- 172.21.15.52:0/3024520318 >> v1:172.21.15.119:6789/0 conn(0x7f589406c480 legacy=0x7f58940cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.230+0000 7f58937fe640 1 -- 172.21.15.52:0/3024520318 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58941607f0 con 0x7f589415c0d0 2024-05-11T01:01:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.230+0000 7f58937fe640 1 --2- 172.21.15.52:0/3024520318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589415c0d0 0x7f5894072a00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58940cb960 tx=0x7f588402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb7170c3db9d749 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:59.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.230+0000 7f5892ffd640 1 -- 172.21.15.52:0/3024520318 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884033640 con 0x7f589415c0d0 2024-05-11T01:01:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5892ffd640 1 -- 172.21.15.52:0/3024520318 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5884038070 con 0x7f589415c0d0 2024-05-11T01:01:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5892ffd640 1 -- 172.21.15.52:0/3024520318 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884033a00 con 0x7f589415c0d0 2024-05-11T01:01:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5899a6d640 1 -- 172.21.15.52:0/3024520318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589415c0d0 msgr2=0x7f5894072a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5899a6d640 1 --2- 172.21.15.52:0/3024520318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589415c0d0 0x7f5894072a00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58940cb960 tx=0x7f588402c8b0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5899a6d640 1 -- 172.21.15.52:0/3024520318 shutdown_connections 2024-05-11T01:01:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5899a6d640 1 --2- 172.21.15.52:0/3024520318 >> v2:172.21.15.119:3300/0 conn(0x7f589406e880 0x7f589415fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5899a6d640 1 --2- 172.21.15.52:0/3024520318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589415c0d0 0x7f5894072a00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5899a6d640 1 -- 172.21.15.52:0/3024520318 >> 172.21.15.52:0/3024520318 conn(0x7f58940cd830 msgr2=0x7f58940cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5899a6d640 1 -- 172.21.15.52:0/3024520318 shutdown_connections 2024-05-11T01:01:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.231+0000 7f5899a6d640 1 -- 172.21.15.52:0/3024520318 wait complete. 2024-05-11T01:01:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.232+0000 7f5899a6d640 1 Processor -- start 2024-05-11T01:01:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f5899a6d640 1 -- start start 2024-05-11T01:01:59.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f5899a6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589406c480 0x7f5894154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f5899a6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589406e880 0x7f58940c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f5899a6d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f589415c0d0 0x7f58940b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f5899a6d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5894072f40 con 0x7f589406c480 2024-05-11T01:01:59.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f5899a6d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58940cc490 con 0x7f589406e880 2024-05-11T01:01:59.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f5899a6d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f58940730c0 con 0x7f589415c0d0 2024-05-11T01:01:59.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f58937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589406c480 0x7f5894154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f58937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589406c480 0x7f5894154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47576/0 (socket says 172.21.15.52:47576) 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.233+0000 7f58937fe640 1 -- 172.21.15.52:0/72444343 learned_addr learned my addr 172.21.15.52:0/72444343 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f588affd640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589406e880 0x7f58940c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f5893fff640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f589415c0d0 0x7f58940b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f58937fe640 1 -- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f589415c0d0 msgr2=0x7f58940b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f58937fe640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f589415c0d0 0x7f58940b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f58937fe640 1 -- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589406e880 msgr2=0x7f58940c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f58937fe640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589406e880 0x7f58940c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f58937fe640 1 -- 172.21.15.52:0/72444343 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58940ca440 con 0x7f589406c480 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f588affd640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589406e880 0x7f58940c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:01:59.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f58937fe640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589406c480 0x7f5894154e40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f588402c8f0 tx=0x7f5884004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f5890ff9640 1 -- 172.21.15.52:0/72444343 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884043070 con 0x7f589406c480 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f5890ff9640 1 -- 172.21.15.52:0/72444343 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588403c830 con 0x7f589406c480 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58940ca720 con 0x7f589406c480 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f5890ff9640 1 -- 172.21.15.52:0/72444343 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588403cc20 con 0x7f589406c480 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.234+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58941e5b00 con 0x7f589406c480 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.235+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58940cac40 con 0x7f589406c480 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.236+0000 7f5890ff9640 1 -- 172.21.15.52:0/72444343 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 98056+0+0 (secure 0 0 0) 0x7f5884046070 con 0x7f589406c480 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.236+0000 7f5890ff9640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5874076170 0x7f5874078630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.236+0000 7f5890ff9640 1 -- 172.21.15.52:0/72444343 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f58840b7d10 con 0x7f589406c480 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.236+0000 7f588affd640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5874076170 0x7f5874078630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.237+0000 7f588affd640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5874076170 0x7f5874078630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f58940b0cc0 tx=0x7f587c008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:59.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.241+0000 7f5890ff9640 1 -- 172.21.15.52:0/72444343 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f58940cac40 con 0x7f589406c480 2024-05-11T01:01:59.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.456+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f58940695d0 con 0x7f5874076170 2024-05-11T01:01:59.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.525+0000 7f5890ff9640 1 -- 172.21.15.52:0/72444343 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f58940695d0 con 0x7f5874076170 2024-05-11T01:01:59.526 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir1 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.527+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5874076170 msgr2=0x7f5874078630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.527+0000 7f5899a6d640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5874076170 0x7f5874078630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f58940b0cc0 tx=0x7f587c008040 comp rx=0 tx=0).stop 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.527+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589406c480 msgr2=0x7f5894154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.527+0000 7f5899a6d640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589406c480 0x7f5894154e40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f588402c8f0 tx=0x7f5884004730 comp rx=0 tx=0).stop 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.527+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 shutdown_connections 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.528+0000 7f5899a6d640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f589415c0d0 0x7f58940b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.528+0000 7f5899a6d640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589406e880 0x7f58940c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.528+0000 7f5899a6d640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5874076170 0x7f5874078630 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.528+0000 7f5899a6d640 1 --2- 172.21.15.52:0/72444343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f589406c480 0x7f5894154e40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.528+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 >> 172.21.15.52:0/72444343 conn(0x7f58940cd830 msgr2=0x7f58940b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.528+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 shutdown_connections 2024-05-11T01:01:59.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.528+0000 7f5899a6d640 1 -- 172.21.15.52:0/72444343 wait complete. 2024-05-11T01:01:59.539 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2024-05-11T01:01:59.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:01:59.539 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2024-05-11T01:01:59.557 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1m 2024-05-11T01:01:59.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.704+0000 7f74ef532640 1 Processor -- start 2024-05-11T01:01:59.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.704+0000 7f74ef532640 1 -- start start 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.705+0000 7f74ef532640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f74e80b28d0 0x7f74e80d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.705+0000 7f74ef532640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74e80b2d20 con 0x7f74e8160bc0 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.705+0000 7f74ef532640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74e80ce7a0 con 0x7f74e81607e0 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.705+0000 7f74ef532640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f74e80d0b40 con 0x7f74e80b28d0 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.705+0000 7f74edacf640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f74e8160bc0 0x7f74e8160fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51148/0 (socket says 172.21.15.52:51148) 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.705+0000 7f74edacf640 1 -- 172.21.15.52:0/3984920587 learned_addr learned my addr 172.21.15.52:0/3984920587 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.705+0000 7f74ecacd640 1 --2- 172.21.15.52:0/3984920587 >> v2:172.21.15.119:3300/0 conn(0x7f74e80b28d0 0x7f74e80d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.705+0000 7f74e7fff640 1 -- 172.21.15.52:0/3984920587 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119513498 0 0) 0x7f74e80b2d20 con 0x7f74e8160bc0 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.705+0000 7f74e7fff640 1 -- 172.21.15.52:0/3984920587 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74d0003520 con 0x7f74e8160bc0 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.706+0000 7f74e7fff640 1 -- 172.21.15.52:0/3984920587 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2212139526 0 0) 0x7f74e80ce7a0 con 0x7f74e81607e0 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.706+0000 7f74e7fff640 1 -- 172.21.15.52:0/3984920587 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74e80b2d20 con 0x7f74e81607e0 2024-05-11T01:01:59.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.706+0000 7f74e7fff640 1 -- 172.21.15.52:0/3984920587 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2894344827 0 0) 0x7f74d0003520 con 0x7f74e8160bc0 2024-05-11T01:01:59.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.706+0000 7f74ecacd640 1 -- 172.21.15.52:0/3984920587 >> v1:172.21.15.119:6789/0 conn(0x7f74e81607e0 legacy=0x7f74e80d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.706+0000 7f74ecacd640 1 -- 172.21.15.52:0/3984920587 >> v1:172.21.15.52:6789/0 conn(0x7f74e8160bc0 legacy=0x7f74e8160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.706+0000 7f74ecacd640 1 -- 172.21.15.52:0/3984920587 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e80cea80 con 0x7f74e80b28d0 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74ecacd640 1 --2- 172.21.15.52:0/3984920587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e80b28d0 0x7f74e80d19c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0097a0 tx=0x7f74dc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d2aa2eeffb52eb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74e7fff640 1 -- 172.21.15.52:0/3984920587 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc037070 con 0x7f74e80b28d0 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74e7fff640 1 -- 172.21.15.52:0/3984920587 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74dc02ace0 con 0x7f74e80b28d0 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74e7fff640 1 -- 172.21.15.52:0/3984920587 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8001a70 con 0x7f74e80b28d0 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74ef532640 1 -- 172.21.15.52:0/3984920587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e80b28d0 msgr2=0x7f74e80d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74ef532640 1 --2- 172.21.15.52:0/3984920587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e80b28d0 0x7f74e80d19c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0097a0 tx=0x7f74dc02a250 comp rx=0 tx=0).stop 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74ef532640 1 -- 172.21.15.52:0/3984920587 shutdown_connections 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74ef532640 1 --2- 172.21.15.52:0/3984920587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e80b28d0 0x7f74e80d19c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74ef532640 1 -- 172.21.15.52:0/3984920587 >> 172.21.15.52:0/3984920587 conn(0x7f74e80b4a60 msgr2=0x7f74e80b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74ef532640 1 -- 172.21.15.52:0/3984920587 shutdown_connections 2024-05-11T01:01:59.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.707+0000 7f74ef532640 1 -- 172.21.15.52:0/3984920587 wait complete. 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ef532640 1 Processor -- start 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ef532640 1 -- start start 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ef532640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e80b28d0 0x7f74e8151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ef532640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e81607e0 0x7f74e815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ef532640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f74e8160bc0 0x7f74e80c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ef532640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f74e80d1f00 con 0x7f74e80b28d0 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ef532640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f74e80d0d90 con 0x7f74e81607e0 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ef532640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f74e80c9d00 con 0x7f74e8160bc0 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ed2ce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e80b28d0 0x7f74e8151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ed2ce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e80b28d0 0x7f74e8151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47586/0 (socket says 172.21.15.52:47586) 2024-05-11T01:01:59.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.709+0000 7f74ed2ce640 1 -- 172.21.15.52:0/3758305534 learned_addr learned my addr 172.21.15.52:0/3758305534 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:01:59.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74ecacd640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e81607e0 0x7f74e815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74edacf640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f74e8160bc0 0x7f74e80c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74ed2ce640 1 -- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f74e8160bc0 msgr2=0x7f74e80c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74ed2ce640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f74e8160bc0 0x7f74e80c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74ed2ce640 1 -- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e81607e0 msgr2=0x7f74e815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74ed2ce640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e81607e0 0x7f74e815b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74ed2ce640 1 -- 172.21.15.52:0/3758305534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e815a6b0 con 0x7f74e80b28d0 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74edacf640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f74e8160bc0 0x7f74e80c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74ed2ce640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e80b28d0 0x7f74e8151d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f74e000c4c0 tx=0x7f74e000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74e5ffb640 1 -- 172.21.15.52:0/3758305534 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e00052d0 con 0x7f74e80b28d0 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74e5ffb640 1 -- 172.21.15.52:0/3758305534 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e0005490 con 0x7f74e80b28d0 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e815a990 con 0x7f74e80b28d0 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74e5ffb640 1 -- 172.21.15.52:0/3758305534 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0012870 con 0x7f74e80b28d0 2024-05-11T01:01:59.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.710+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e815af50 con 0x7f74e80b28d0 2024-05-11T01:01:59.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.711+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e80695d0 con 0x7f74e80b28d0 2024-05-11T01:01:59.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.712+0000 7f74e5ffb640 1 -- 172.21.15.52:0/3758305534 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 98056+0+0 (secure 0 0 0) 0x7f74e0012a10 con 0x7f74e80b28d0 2024-05-11T01:01:59.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.712+0000 7f74e5ffb640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f74d40761a0 0x7f74d4078660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:01:59.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.713+0000 7f74e5ffb640 1 -- 172.21.15.52:0/3758305534 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f74e0096eb0 con 0x7f74e80b28d0 2024-05-11T01:01:59.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.713+0000 7f74ecacd640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f74d40761a0 0x7f74d4078660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:01:59.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.713+0000 7f74ecacd640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f74d40761a0 0x7f74d4078660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f74dc034000 tx=0x7f74dc02e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:01:59.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.717+0000 7f74e5ffb640 1 -- 172.21.15.52:0/3758305534 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f74e0060a50 con 0x7f74e80b28d0 2024-05-11T01:01:59.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.929+0000 7f74e5ffb640 1 -- 172.21.15.52:0/3758305534 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 9) ==== 98321+0+0 (secure 0 0 0) 0x7f74e005c5e0 con 0x7f74e80b28d0 2024-05-11T01:01:59.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.936+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f74e8072260 con 0x7f74d40761a0 2024-05-11T01:01:59.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.993+0000 7f74e5ffb640 1 -- 172.21.15.52:0/3758305534 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f74e8072260 con 0x7f74d40761a0 2024-05-11T01:01:59.993 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir2 2024-05-11T01:01:59.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f74d40761a0 msgr2=0x7f74d4078660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f74d40761a0 0x7f74d4078660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f74dc034000 tx=0x7f74dc02e000 comp rx=0 tx=0).stop 2024-05-11T01:01:59.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e80b28d0 msgr2=0x7f74e8151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:01:59.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e80b28d0 0x7f74e8151d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f74e000c4c0 tx=0x7f74e000c980 comp rx=0 tx=0).stop 2024-05-11T01:01:59.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 shutdown_connections 2024-05-11T01:01:59.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f74e8160bc0 0x7f74e80c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74e81607e0 0x7f74e815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f74d40761a0 0x7f74d4078660 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 --2- 172.21.15.52:0/3758305534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e80b28d0 0x7f74e8151d60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:01:59.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 >> 172.21.15.52:0/3758305534 conn(0x7f74e80b4a60 msgr2=0x7f74e80d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:01:59.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 shutdown_connections 2024-05-11T01:01:59.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:01:59.995+0000 7f74ef532640 1 -- 172.21.15.52:0/3758305534 wait complete. 2024-05-11T01:02:00.006 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2024-05-11T01:02:00.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:02:00.006 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2024-05-11T01:02:00.025 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1m 2024-05-11T01:02:00.051 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:02:00.051+0000 7f89cd345640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:02:00.051 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:02:00.051+0000 7f89cd345640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:02:00.052 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:02:00.051+0000 7f89cd345640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:02:00.052 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:02:00.051+0000 7f89cd345640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:02:00.052 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:02:00.051+0000 7f89cd345640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:02:00.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.178+0000 7ffa528dc640 1 Processor -- start 2024-05-11T01:02:00.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.178+0000 7ffa528dc640 1 -- start start 2024-05-11T01:02:00.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa528dc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa4c15f2f0 0x7ffa4c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa528dc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c0d4e40 0x7ffa4c0d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa528dc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffa4c0d2d60 0x7ffa4c0d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa528dc640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ffa4c0d57f0 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa528dc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffa4c0d3a90 con 0x7ffa4c15f2f0 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa528dc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4c0d3d70 con 0x7ffa4c0d4e40 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa50e79640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffa4c0d2d60 0x7ffa4c0d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa50e79640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffa4c0d2d60 0x7ffa4c0d31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47598/0 (socket says 172.21.15.52:47598) 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa50e79640 1 -- 172.21.15.52:0/2505437048 learned_addr learned my addr 172.21.15.52:0/2505437048 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa43fff640 1 --2- 172.21.15.52:0/2505437048 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c0d4e40 0x7ffa4c0d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa4bfff640 1 --2- 172.21.15.52:0/2505437048 >> v2:172.21.15.119:3300/0 conn(0x7ffa4c15f2f0 0x7ffa4c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa50e79640 1 -- 172.21.15.52:0/2505437048 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c0d4e40 msgr2=0x7ffa4c0d52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa50e79640 1 --2- 172.21.15.52:0/2505437048 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c0d4e40 0x7ffa4c0d52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa50e79640 1 -- 172.21.15.52:0/2505437048 >> v2:172.21.15.119:3300/0 conn(0x7ffa4c15f2f0 msgr2=0x7ffa4c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:00.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa50e79640 1 --2- 172.21.15.52:0/2505437048 >> v2:172.21.15.119:3300/0 conn(0x7ffa4c15f2f0 0x7ffa4c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.179+0000 7ffa50e79640 1 -- 172.21.15.52:0/2505437048 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c0d4050 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.180+0000 7ffa50e79640 1 --2- 172.21.15.52:0/2505437048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 0x7ffa4c0d31a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0d3ca0 tx=0x7ffa4402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47e037fa83a07af server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:00.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.180+0000 7ffa4b7fe640 1 -- 172.21.15.52:0/2505437048 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44036070 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.180+0000 7ffa4b7fe640 1 -- 172.21.15.52:0/2505437048 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa44031600 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.180+0000 7ffa4b7fe640 1 -- 172.21.15.52:0/2505437048 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440319d0 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.180+0000 7ffa528dc640 1 -- 172.21.15.52:0/2505437048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 msgr2=0x7ffa4c0d31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.180+0000 7ffa528dc640 1 --2- 172.21.15.52:0/2505437048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 0x7ffa4c0d31a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0d3ca0 tx=0x7ffa4402a680 comp rx=0 tx=0).stop 2024-05-11T01:02:00.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.181+0000 7ffa528dc640 1 -- 172.21.15.52:0/2505437048 shutdown_connections 2024-05-11T01:02:00.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.181+0000 7ffa528dc640 1 --2- 172.21.15.52:0/2505437048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 0x7ffa4c0d31a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.181+0000 7ffa528dc640 1 --2- 172.21.15.52:0/2505437048 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c0d4e40 0x7ffa4c0d52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.181+0000 7ffa528dc640 1 --2- 172.21.15.52:0/2505437048 >> v2:172.21.15.119:3300/0 conn(0x7ffa4c15f2f0 0x7ffa4c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.181+0000 7ffa528dc640 1 -- 172.21.15.52:0/2505437048 >> 172.21.15.52:0/2505437048 conn(0x7ffa4c08ebb0 msgr2=0x7ffa4c09e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:00.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.181+0000 7ffa528dc640 1 -- 172.21.15.52:0/2505437048 shutdown_connections 2024-05-11T01:02:00.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.181+0000 7ffa528dc640 1 -- 172.21.15.52:0/2505437048 wait complete. 2024-05-11T01:02:00.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.182+0000 7ffa528dc640 1 Processor -- start 2024-05-11T01:02:00.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.182+0000 7ffa528dc640 1 -- start start 2024-05-11T01:02:00.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.182+0000 7ffa528dc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 0x7ffa4c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa528dc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa4c0d4e40 0x7ffa4c15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa528dc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c15f2f0 0x7ffa4c161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa528dc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c0d36e0 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa528dc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c0b5140 con 0x7ffa4c0d4e40 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa528dc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4c15aa00 con 0x7ffa4c15f2f0 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 0x7ffa4c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 0x7ffa4c0a2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47612/0 (socket says 172.21.15.52:47612) 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa4bfff640 1 -- 172.21.15.52:0/4240762029 learned_addr learned my addr 172.21.15.52:0/4240762029 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa50e79640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c15f2f0 0x7ffa4c161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa43fff640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa4c0d4e40 0x7ffa4c15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa4bfff640 1 -- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c15f2f0 msgr2=0x7ffa4c161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa4bfff640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c15f2f0 0x7ffa4c161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa4bfff640 1 -- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa4c0d4e40 msgr2=0x7ffa4c15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa4bfff640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa4c0d4e40 0x7ffa4c15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa4bfff640 1 -- 172.21.15.52:0/4240762029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c1626e0 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa4bfff640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 0x7ffa4c0a2a70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00d800 tx=0x7ffa3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa497fa640 1 -- 172.21.15.52:0/4240762029 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00b6c0 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa497fa640 1 -- 172.21.15.52:0/4240762029 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3c019070 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c0cd310 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa497fa640 1 -- 172.21.15.52:0/4240762029 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00ba80 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.183+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c1629c0 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.184+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4c0695d0 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.185+0000 7ffa497fa640 1 -- 172.21.15.52:0/4240762029 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 98321+0+0 (secure 0 0 0) 0x7ffa3c00bc20 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.185+0000 7ffa497fa640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffa2c076440 0x7ffa2c078900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.186+0000 7ffa497fa640 1 -- 172.21.15.52:0/4240762029 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7ffa3c097b50 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.186+0000 7ffa43fff640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffa2c076440 0x7ffa2c078900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.186+0000 7ffa43fff640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffa2c076440 0x7ffa2c078900 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0b2d20 tx=0x7ffa34008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:00.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.190+0000 7ffa497fa640 1 -- 172.21.15.52:0/4240762029 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffa3c060ce0 con 0x7ffa4c0d2d60 2024-05-11T01:02:00.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.404+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ffa4c0d6080 con 0x7ffa2c076440 2024-05-11T01:02:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.458+0000 7ffa497fa640 1 -- 172.21.15.52:0/4240762029 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ffa4c0d6080 con 0x7ffa2c076440 2024-05-11T01:02:00.459 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir3 2024-05-11T01:02:00.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffa2c076440 msgr2=0x7ffa2c078900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffa2c076440 0x7ffa2c078900 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0b2d20 tx=0x7ffa34008040 comp rx=0 tx=0).stop 2024-05-11T01:02:00.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 msgr2=0x7ffa4c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 0x7ffa4c0a2a70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00d800 tx=0x7ffa3c00dcc0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 shutdown_connections 2024-05-11T01:02:00.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffa4c15f2f0 0x7ffa4c161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa4c0d4e40 0x7ffa4c15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffa2c076440 0x7ffa2c078900 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 --2- 172.21.15.52:0/4240762029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa4c0d2d60 0x7ffa4c0a2a70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.460+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 >> 172.21.15.52:0/4240762029 conn(0x7ffa4c08ebb0 msgr2=0x7ffa4c06dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:00.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.461+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 shutdown_connections 2024-05-11T01:02:00.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.461+0000 7ffa528dc640 1 -- 172.21.15.52:0/4240762029 wait complete. 2024-05-11T01:02:00.471 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2024-05-11T01:02:00.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:02:00.472 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2024-05-11T01:02:00.490 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1m 2024-05-11T01:02:00.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.645+0000 7f4966012640 1 Processor -- start 2024-05-11T01:02:00.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.645+0000 7f4966012640 1 -- start start 2024-05-11T01:02:00.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.645+0000 7f4966012640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49600b4450 0x7f49600ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.645+0000 7f4966012640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49600d1150 con 0x7f496015d4b0 2024-05-11T01:02:00.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.645+0000 7f4966012640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49600b2680 con 0x7f49600d1b40 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.645+0000 7f4966012640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f49600b2960 con 0x7f49600b4450 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495f7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f496015d4b0 0x7f49600d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:51150/0 (socket says 172.21.15.52:51150) 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495f7fe640 1 -- 172.21.15.52:0/299894180 learned_addr learned my addr 172.21.15.52:0/299894180 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495ffff640 1 --2- 172.21.15.52:0/299894180 >> v2:172.21.15.119:3300/0 conn(0x7f49600b4450 0x7f49600ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3935553881 0 0) 0x7f49600d1150 con 0x7f496015d4b0 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4948003520 con 0x7f496015d4b0 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2205925605 0 0) 0x7f4948003520 con 0x7f496015d4b0 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 >> v2:172.21.15.119:3300/0 conn(0x7f49600b4450 msgr2=0x7f49600ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 --2- 172.21.15.52:0/299894180 >> v2:172.21.15.119:3300/0 conn(0x7f49600b4450 0x7f49600ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 >> v1:172.21.15.119:6789/0 conn(0x7f49600d1b40 legacy=0x7f49600cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49600b2c40 con 0x7f496015d4b0 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f49500016f0 con 0x7f496015d4b0 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495ffff640 1 --2- 172.21.15.52:0/299894180 >> v2:172.21.15.119:3300/0 conn(0x7f49600b4450 0x7f49600ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:02:00.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 >> v1:172.21.15.52:6789/0 conn(0x7f496015d4b0 legacy=0x7f49600d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4948003a50 0x7f4948003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4948004800 0x7f4948004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49480052e0 0x7f494800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f49600b2960 con 0x7f4948004800 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f494800c1b0 con 0x7f49480052e0 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f494800c3b0 con 0x7f4948003a50 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495effd640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4948004800 0x7f4948004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495f7fe640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4948003a50 0x7f4948003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.646+0000 7f495ffff640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49480052e0 0x7f494800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.647+0000 7f495effd640 1 -- 172.21.15.52:0/299894180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4948003a50 msgr2=0x7f4948003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.647+0000 7f495effd640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4948003a50 0x7f4948003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.647+0000 7f495effd640 1 -- 172.21.15.52:0/299894180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49480052e0 msgr2=0x7f494800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.647+0000 7f495effd640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49480052e0 0x7f494800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.647+0000 7f495effd640 1 -- 172.21.15.52:0/299894180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4948003520 con 0x7f4948004800 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.647+0000 7f495ffff640 1 -- 172.21.15.52:0/299894180 reap_dead start 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.647+0000 7f495effd640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4948004800 0x7f4948004c80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4954001760 tx=0x7f4948005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d156a17fd7cab13 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.647+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49540013b0 con 0x7f4948004800 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.648+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49540052d0 con 0x7f4948004800 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.648+0000 7f495e7fc640 1 -- 172.21.15.52:0/299894180 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495402aa50 con 0x7f4948004800 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.648+0000 7f4966012640 1 -- 172.21.15.52:0/299894180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4948004800 msgr2=0x7f4948004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.648+0000 7f4966012640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4948004800 0x7f4948004c80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4954001760 tx=0x7f4948005830 comp rx=0 tx=0).stop 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.648+0000 7f4966012640 1 -- 172.21.15.52:0/299894180 shutdown_connections 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.648+0000 7f4966012640 1 --2- 172.21.15.52:0/299894180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4948004800 0x7f4948004c80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.648+0000 7f4966012640 1 -- 172.21.15.52:0/299894180 >> 172.21.15.52:0/299894180 conn(0x7f49600d3000 msgr2=0x7f49600d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.648+0000 7f4966012640 1 -- 172.21.15.52:0/299894180 shutdown_connections 2024-05-11T01:02:00.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.648+0000 7f4966012640 1 -- 172.21.15.52:0/299894180 wait complete. 2024-05-11T01:02:00.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.649+0000 7f4966012640 1 Processor -- start 2024-05-11T01:02:00.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.649+0000 7f4966012640 1 -- start start 2024-05-11T01:02:00.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f4966012640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4948004800 0x7f49601573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f4966012640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4960157910 0x7f49601f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f4966012640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49601f6d60 0x7f496014ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f4966012640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f496015d890 con 0x7f49601f6d60 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f4966012640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f49600d0de0 con 0x7f4948004800 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f4966012640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f49600caf00 con 0x7f4960157910 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f495ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49601f6d60 0x7f496014ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f495ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49601f6d60 0x7f496014ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47638/0 (socket says 172.21.15.52:47638) 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f495ffff640 1 -- 172.21.15.52:0/4294285844 learned_addr learned my addr 172.21.15.52:0/4294285844 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f495f7fe640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4948004800 0x7f49601573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.650+0000 7f495effd640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4960157910 0x7f49601f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f495ffff640 1 -- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4960157910 msgr2=0x7f49601f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f495ffff640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4960157910 0x7f49601f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f495ffff640 1 -- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4948004800 msgr2=0x7f49601573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f495ffff640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4948004800 0x7f49601573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f495ffff640 1 -- 172.21.15.52:0/4294285844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49600cb1e0 con 0x7f49601f6d60 2024-05-11T01:02:00.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f495ffff640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49601f6d60 0x7f496014ab30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49600cf370 tx=0x7f495800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f4947fff640 1 -- 172.21.15.52:0/4294285844 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800fdf0 con 0x7f49601f6d60 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f4947fff640 1 -- 172.21.15.52:0/4294285844 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4958003990 con 0x7f49601f6d60 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f4947fff640 1 -- 172.21.15.52:0/4294285844 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958003d80 con 0x7f49601f6d60 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49600cb4c0 con 0x7f49601f6d60 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.651+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49600caae0 con 0x7f49601f6d60 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.652+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49600cb9e0 con 0x7f49601f6d60 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.653+0000 7f4947fff640 1 -- 172.21.15.52:0/4294285844 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 98321+0+0 (secure 0 0 0) 0x7f495801f020 con 0x7f49601f6d60 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.653+0000 7f4947fff640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f494c076440 0x7f494c078900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.653+0000 7f4947fff640 1 -- 172.21.15.52:0/4294285844 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f4958097e60 con 0x7f49601f6d60 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.653+0000 7f495f7fe640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f494c076440 0x7f494c078900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:00.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.654+0000 7f495f7fe640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f494c076440 0x7f494c078900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4950001220 tx=0x7f4950007970 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:00.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.657+0000 7f4947fff640 1 -- 172.21.15.52:0/4294285844 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f49600cb9e0 con 0x7f49601f6d60 2024-05-11T01:02:00.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.872+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4960069c00 con 0x7f494c076440 2024-05-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.935+0000 7f4947fff640 1 -- 172.21.15.52:0/4294285844 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f4960069c00 con 0x7f494c076440 2024-05-11T01:02:00.935 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir4 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f494c076440 msgr2=0x7f494c078900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f494c076440 0x7f494c078900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4950001220 tx=0x7f4950007970 comp rx=0 tx=0).stop 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49601f6d60 msgr2=0x7f496014ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49601f6d60 0x7f496014ab30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49600cf370 tx=0x7f495800f580 comp rx=0 tx=0).stop 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 shutdown_connections 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f494c076440 0x7f494c078900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49601f6d60 0x7f496014ab30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4960157910 0x7f49601f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 --2- 172.21.15.52:0/4294285844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4948004800 0x7f49601573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 >> 172.21.15.52:0/4294285844 conn(0x7f49600d3000 msgr2=0x7f49600703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.937+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 shutdown_connections 2024-05-11T01:02:00.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:00.938+0000 7f4966012640 1 -- 172.21.15.52:0/4294285844 wait complete. 2024-05-11T01:02:00.948 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2024-05-11T01:02:00.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:02:00.949 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2024-05-11T01:02:00.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1m 2024-05-11T01:02:01.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.121+0000 7f28ec9f2640 1 Processor -- start 2024-05-11T01:02:01.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.121+0000 7f28ec9f2640 1 -- start start 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.121+0000 7f28ec9f2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3940 0x7f28e806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.121+0000 7f28ec9f2640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f28e815e0d0 0x7f28e80b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.121+0000 7f28ec9f2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e8160d00 con 0x7f28e80b3d20 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28ec9f2640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f28e806f8e0 con 0x7f28e815e0d0 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28ec9f2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f28e8160fc0 con 0x7f28e80b3940 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28e6d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f28e815e0d0 0x7f28e80b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28e6d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f28e815e0d0 0x7f28e80b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47650/0 (socket says 172.21.15.52:47650) 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28e6d9d640 1 -- 172.21.15.52:0/2676303140 learned_addr learned my addr 172.21.15.52:0/2676303140 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28e659c640 1 --2- 172.21.15.52:0/2676303140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3940 0x7f28e806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28e6d9d640 1 -- 172.21.15.52:0/2676303140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3940 msgr2=0x7f28e806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28e6d9d640 1 --2- 172.21.15.52:0/2676303140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3940 0x7f28e806f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28e6d9d640 1 -- 172.21.15.52:0/2676303140 >> v1:172.21.15.119:6789/0 conn(0x7f28e80b3d20 legacy=0x7f28e815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.122+0000 7f28e6d9d640 1 -- 172.21.15.52:0/2676303140 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e81612a0 con 0x7f28e815e0d0 2024-05-11T01:02:01.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28e6d9d640 1 --2- 172.21.15.52:0/2676303140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e815e0d0 0x7f28e80b2500 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009e50 tx=0x7f28e002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979b1c9a327bbb37 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:01.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28e5d9b640 1 -- 172.21.15.52:0/2676303140 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0036070 con 0x7f28e815e0d0 2024-05-11T01:02:01.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28e5d9b640 1 -- 172.21.15.52:0/2676303140 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e0031600 con 0x7f28e815e0d0 2024-05-11T01:02:01.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28e5d9b640 1 -- 172.21.15.52:0/2676303140 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e00319d0 con 0x7f28e815e0d0 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2676303140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e815e0d0 msgr2=0x7f28e80b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28ec9f2640 1 --2- 172.21.15.52:0/2676303140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e815e0d0 0x7f28e80b2500 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009e50 tx=0x7f28e002a710 comp rx=0 tx=0).stop 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2676303140 shutdown_connections 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28ec9f2640 1 --2- 172.21.15.52:0/2676303140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e815e0d0 0x7f28e80b2500 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28ec9f2640 1 --2- 172.21.15.52:0/2676303140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3940 0x7f28e806f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2676303140 >> 172.21.15.52:0/2676303140 conn(0x7f28e8075ca0 msgr2=0x7f28e8076080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2676303140 shutdown_connections 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.123+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2676303140 wait complete. 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.125+0000 7f28ec9f2640 1 Processor -- start 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.125+0000 7f28ec9f2640 1 -- start start 2024-05-11T01:02:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.125+0000 7f28ec9f2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80b3940 0x7f28e80aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.125+0000 7f28ec9f2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3d20 0x7f28e80c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28ec9f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28e815e0d0 0x7f28e80c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28ec9f2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28e8161b40 con 0x7f28e80b3940 2024-05-11T01:02:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28ec9f2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f28e8161860 con 0x7f28e815e0d0 2024-05-11T01:02:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28ec9f2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f28e815e510 con 0x7f28e80b3d20 2024-05-11T01:02:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80b3940 0x7f28e80aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80b3940 0x7f28e80aa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47662/0 (socket says 172.21.15.52:47662) 2024-05-11T01:02:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e659c640 1 -- 172.21.15.52:0/2366616885 learned_addr learned my addr 172.21.15.52:0/2366616885 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28dffff640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3d20 0x7f28e80c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e6d9d640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28e815e0d0 0x7f28e80c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e659c640 1 -- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3d20 msgr2=0x7f28e80c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e659c640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3d20 0x7f28e80c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e659c640 1 -- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28e815e0d0 msgr2=0x7f28e80c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e659c640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28e815e0d0 0x7f28e80c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e659c640 1 -- 172.21.15.52:0/2366616885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e80cb140 con 0x7f28e80b3940 2024-05-11T01:02:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28e659c640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80b3940 0x7f28e80aa480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f28d800e9b0 tx=0x7f28d800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28deffd640 1 -- 172.21.15.52:0/2366616885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8017070 con 0x7f28e80b3940 2024-05-11T01:02:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28deffd640 1 -- 172.21.15.52:0/2366616885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d800c9a0 con 0x7f28e80b3940 2024-05-11T01:02:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e80cb420 con 0x7f28e80b3940 2024-05-11T01:02:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28deffd640 1 -- 172.21.15.52:0/2366616885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d800cd90 con 0x7f28e80b3940 2024-05-11T01:02:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.126+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e80cbcc0 con 0x7f28e80b3940 2024-05-11T01:02:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.127+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e80695d0 con 0x7f28e80b3940 2024-05-11T01:02:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.128+0000 7f28deffd640 1 -- 172.21.15.52:0/2366616885 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 98321+0+0 (secure 0 0 0) 0x7f28d801a070 con 0x7f28e80b3940 2024-05-11T01:02:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.129+0000 7f28deffd640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f28c8076440 0x7f28c8078900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.129+0000 7f28deffd640 1 -- 172.21.15.52:0/2366616885 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f28d8097c40 con 0x7f28e80b3940 2024-05-11T01:02:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.129+0000 7f28dffff640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f28c8076440 0x7f28c8078900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.129+0000 7f28dffff640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f28c8076440 0x7f28c8078900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f28e80c6900 tx=0x7f28d0009040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.133+0000 7f28deffd640 1 -- 172.21.15.52:0/2366616885 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f28d8060d40 con 0x7f28e80b3940 2024-05-11T01:02:01.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.349+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f28e8072af0 con 0x7f28c8076440 2024-05-11T01:02:01.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.409+0000 7f28deffd640 1 -- 172.21.15.52:0/2366616885 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f28e8072af0 con 0x7f28c8076440 2024-05-11T01:02:01.409 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir5 2024-05-11T01:02:01.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f28c8076440 msgr2=0x7f28c8078900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f28c8076440 0x7f28c8078900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f28e80c6900 tx=0x7f28d0009040 comp rx=0 tx=0).stop 2024-05-11T01:02:01.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80b3940 msgr2=0x7f28e80aa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80b3940 0x7f28e80aa480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f28d800e9b0 tx=0x7f28d800ee70 comp rx=0 tx=0).stop 2024-05-11T01:02:01.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 shutdown_connections 2024-05-11T01:02:01.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28e815e0d0 0x7f28e80c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f28e80b3d20 0x7f28e80c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f28c8076440 0x7f28c8078900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 --2- 172.21.15.52:0/2366616885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80b3940 0x7f28e80aa480 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 >> 172.21.15.52:0/2366616885 conn(0x7f28e8075ca0 msgr2=0x7f28e80cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:01.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 shutdown_connections 2024-05-11T01:02:01.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.411+0000 7f28ec9f2640 1 -- 172.21.15.52:0/2366616885 wait complete. 2024-05-11T01:02:01.424 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2024-05-11T01:02:01.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:02:01.424 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2024-05-11T01:02:01.448 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1m 2024-05-11T01:02:01.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.595+0000 7f0ebcbf0640 1 Processor -- start 2024-05-11T01:02:01.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.595+0000 7f0ebcbf0640 1 -- start start 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0ebcbf0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0eb80b4320 0x7f0eb806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0ebcbf0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0eb80d2b40 0x7f0eb809ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0ebcbf0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2380 0x7f0eb809bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0ebcbf0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f0eb806dd80 con 0x7f0eb80d2b40 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0ebcbf0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0eb806e140 con 0x7f0eb80b4320 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0ebcbf0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb806e420 con 0x7f0eb80d2380 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0eb5d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0eb80d2b40 0x7f0eb809ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0eb5d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f0eb80d2b40 0x7f0eb809ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47666/0 (socket says 172.21.15.52:47666) 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0eb5d9b640 1 -- 172.21.15.52:0/105569998 learned_addr learned my addr 172.21.15.52:0/105569998 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0eb6d9d640 1 --2- 172.21.15.52:0/105569998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2380 0x7f0eb809bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.596+0000 7f0eb659c640 1 --2- 172.21.15.52:0/105569998 >> v2:172.21.15.119:3300/0 conn(0x7f0eb80b4320 0x7f0eb806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.597+0000 7f0eb5d9b640 1 -- 172.21.15.52:0/105569998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2380 msgr2=0x7f0eb809bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:01.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.597+0000 7f0eb5d9b640 1 --2- 172.21.15.52:0/105569998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2380 0x7f0eb809bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.597+0000 7f0eb5d9b640 1 -- 172.21.15.52:0/105569998 >> v2:172.21.15.119:3300/0 conn(0x7f0eb80b4320 msgr2=0x7f0eb806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:01.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.597+0000 7f0eb5d9b640 1 --2- 172.21.15.52:0/105569998 >> v2:172.21.15.119:3300/0 conn(0x7f0eb80b4320 0x7f0eb806d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.597+0000 7f0eb5d9b640 1 -- 172.21.15.52:0/105569998 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb80d1e60 con 0x7f0eb80d2b40 2024-05-11T01:02:01.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.597+0000 7f0eb5d9b640 1 --2- 172.21.15.52:0/105569998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2b40 0x7f0eb809ba20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80766c0 tx=0x7f0eac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e13c1750db6d609 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:01.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0eb559a640 1 -- 172.21.15.52:0/105569998 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac037070 con 0x7f0eb80d2b40 2024-05-11T01:02:01.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0eb559a640 1 -- 172.21.15.52:0/105569998 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eac032440 con 0x7f0eb80d2b40 2024-05-11T01:02:01.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0eb559a640 1 -- 172.21.15.52:0/105569998 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac032810 con 0x7f0eb80d2b40 2024-05-11T01:02:01.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/105569998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2b40 msgr2=0x7f0eb809ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/105569998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2b40 0x7f0eb809ba20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80766c0 tx=0x7f0eac02a140 comp rx=0 tx=0).stop 2024-05-11T01:02:01.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/105569998 shutdown_connections 2024-05-11T01:02:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/105569998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2b40 0x7f0eb809ba20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/105569998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2380 0x7f0eb809bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/105569998 >> v2:172.21.15.119:3300/0 conn(0x7f0eb80b4320 0x7f0eb806d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/105569998 >> 172.21.15.52:0/105569998 conn(0x7f0eb80b4a60 msgr2=0x7f0eb80b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/105569998 shutdown_connections 2024-05-11T01:02:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.598+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/105569998 wait complete. 2024-05-11T01:02:01.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0ebcbf0640 1 Processor -- start 2024-05-11T01:02:01.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0ebcbf0640 1 -- start start 2024-05-11T01:02:01.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0ebcbf0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb80b4320 0x7f0eb815f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0ebcbf0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2380 0x7f0eb81f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0ebcbf0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2b40 0x7f0eb8161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0ebcbf0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb80b5980 con 0x7f0eb80d2380 2024-05-11T01:02:01.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0ebcbf0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb80b5800 con 0x7f0eb80b4320 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0ebcbf0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb81631d0 con 0x7f0eb80d2b40 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0eb5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2380 0x7f0eb81f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0eb5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2380 0x7f0eb81f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47682/0 (socket says 172.21.15.52:47682) 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.600+0000 7f0eb5d9b640 1 -- 172.21.15.52:0/1186422685 learned_addr learned my addr 172.21.15.52:0/1186422685 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0eb6d9d640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2b40 0x7f0eb8161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0eb659c640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb80b4320 0x7f0eb815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0eb5d9b640 1 -- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2b40 msgr2=0x7f0eb8161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0eb5d9b640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2b40 0x7f0eb8161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0eb5d9b640 1 -- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb80b4320 msgr2=0x7f0eb815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0eb5d9b640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb80b4320 0x7f0eb815f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0eb5d9b640 1 -- 172.21.15.52:0/1186422685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb815faa0 con 0x7f0eb80d2380 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0eb5d9b640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2380 0x7f0eb81f4950 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000c00 tx=0x7f0eac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0ea6ffd640 1 -- 172.21.15.52:0/1186422685 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac041070 con 0x7f0eb80d2380 2024-05-11T01:02:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0ea6ffd640 1 -- 172.21.15.52:0/1186422685 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eac032e50 con 0x7f0eb80d2380 2024-05-11T01:02:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb815fd80 con 0x7f0eb80d2380 2024-05-11T01:02:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0ea6ffd640 1 -- 172.21.15.52:0/1186422685 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac037040 con 0x7f0eb80d2380 2024-05-11T01:02:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.601+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb8160340 con 0x7f0eb80d2380 2024-05-11T01:02:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.602+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb80695d0 con 0x7f0eb80d2380 2024-05-11T01:02:01.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.603+0000 7f0ea6ffd640 1 -- 172.21.15.52:0/1186422685 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 98321+0+0 (secure 0 0 0) 0x7f0eac044070 con 0x7f0eb80d2380 2024-05-11T01:02:01.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.603+0000 7f0ea6ffd640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f0e98076440 0x7f0e98078900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:01.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.604+0000 7f0ea6ffd640 1 -- 172.21.15.52:0/1186422685 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f0eac0b8120 con 0x7f0eb80d2380 2024-05-11T01:02:01.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.604+0000 7f0eb659c640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f0e98076440 0x7f0e98078900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:01.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.604+0000 7f0eb659c640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f0e98076440 0x7f0e98078900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009ac0 tx=0x7f0ea8009190 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:01.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.608+0000 7f0ea6ffd640 1 -- 172.21.15.52:0/1186422685 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0eac081180 con 0x7f0eb80d2380 2024-05-11T01:02:01.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.819+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0eb80b8ef0 con 0x7f0e98076440 2024-05-11T01:02:01.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.882+0000 7f0ea6ffd640 1 -- 172.21.15.52:0/1186422685 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0eb80b8ef0 con 0x7f0e98076440 2024-05-11T01:02:01.882 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir6 2024-05-11T01:02:01.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f0e98076440 msgr2=0x7f0e98078900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f0e98076440 0x7f0e98078900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009ac0 tx=0x7f0ea8009190 comp rx=0 tx=0).stop 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2380 msgr2=0x7f0eb81f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2380 0x7f0eb81f4950 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000c00 tx=0x7f0eac0046c0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 shutdown_connections 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0eb80d2b40 0x7f0eb8161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f0e98076440 0x7f0e98078900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0eb80d2380 0x7f0eb81f4950 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 --2- 172.21.15.52:0/1186422685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb80b4320 0x7f0eb815f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 >> 172.21.15.52:0/1186422685 conn(0x7f0eb80b4a60 msgr2=0x7f0eb80afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 shutdown_connections 2024-05-11T01:02:01.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:01.884+0000 7f0ebcbf0640 1 -- 172.21.15.52:0/1186422685 wait complete. 2024-05-11T01:02:01.895 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2024-05-11T01:02:01.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:02:01.896 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2024-05-11T01:02:01.916 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1m 2024-05-11T01:02:02.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.064+0000 7f640a140640 1 Processor -- start 2024-05-11T01:02:02.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.064+0000 7f640a140640 1 -- start start 2024-05-11T01:02:02.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f640a140640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f640415f430 0x7f64040b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f640a140640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f64040b5420 0x7f64040d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f640a140640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64040cfc60 con 0x7f64040d1eb0 2024-05-11T01:02:02.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f640a140640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f64040d2a80 con 0x7f640415f430 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f640a140640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6404163120 con 0x7f64040b5420 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f64037fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f640415f430 0x7f64040b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f64037fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f640415f430 0x7f64040b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37006/0 (socket says 172.21.15.52:37006) 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f64037fe640 1 -- 172.21.15.52:0/3173001347 learned_addr learned my addr 172.21.15.52:0/3173001347 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f6403fff640 1 --1- 172.21.15.52:0/3173001347 >> v1:172.21.15.52:6789/0 conn(0x7f64040d1eb0 0x7f64040d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36854/0 (socket says 172.21.15.52:36854) 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f64037fe640 1 -- 172.21.15.52:0/3173001347 >> v2:172.21.15.119:3300/0 conn(0x7f64040b5420 msgr2=0x7f64040d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f64037fe640 1 --2- 172.21.15.52:0/3173001347 >> v2:172.21.15.119:3300/0 conn(0x7f64040b5420 0x7f64040d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f64037fe640 1 -- 172.21.15.52:0/3173001347 >> v1:172.21.15.52:6789/0 conn(0x7f64040d1eb0 legacy=0x7f64040d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f64037fe640 1 -- 172.21.15.52:0/3173001347 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64040d24c0 con 0x7f640415f430 2024-05-11T01:02:02.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.065+0000 7f64027fc640 1 -- 172.21.15.52:0/3173001347 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1296714939 0 0) 0x7f64040cfc60 con 0x7f64040d1eb0 2024-05-11T01:02:02.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.066+0000 7f64037fe640 1 --2- 172.21.15.52:0/3173001347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f640415f430 0x7f64040b4ee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f640408d7f0 tx=0x7f63f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b105c73f20bdc02 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.066+0000 7f64027fc640 1 -- 172.21.15.52:0/3173001347 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4037070 con 0x7f640415f430 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.066+0000 7f64027fc640 1 -- 172.21.15.52:0/3173001347 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f4032440 con 0x7f640415f430 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.066+0000 7f64027fc640 1 -- 172.21.15.52:0/3173001347 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4032810 con 0x7f640415f430 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.067+0000 7f640a140640 1 -- 172.21.15.52:0/3173001347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f640415f430 msgr2=0x7f64040b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.067+0000 7f640a140640 1 --2- 172.21.15.52:0/3173001347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f640415f430 0x7f64040b4ee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f640408d7f0 tx=0x7f63f402a270 comp rx=0 tx=0).stop 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.067+0000 7f640a140640 1 -- 172.21.15.52:0/3173001347 shutdown_connections 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.067+0000 7f640a140640 1 --2- 172.21.15.52:0/3173001347 >> v2:172.21.15.119:3300/0 conn(0x7f64040b5420 0x7f64040d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.067+0000 7f640a140640 1 --2- 172.21.15.52:0/3173001347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f640415f430 0x7f64040b4ee0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.067+0000 7f640a140640 1 -- 172.21.15.52:0/3173001347 >> 172.21.15.52:0/3173001347 conn(0x7f640408ebb0 msgr2=0x7f640409e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.067+0000 7f640a140640 1 -- 172.21.15.52:0/3173001347 shutdown_connections 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.067+0000 7f640a140640 1 -- 172.21.15.52:0/3173001347 wait complete. 2024-05-11T01:02:02.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.068+0000 7f640a140640 1 Processor -- start 2024-05-11T01:02:02.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.068+0000 7f640a140640 1 -- start start 2024-05-11T01:02:02.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f640a140640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64040b5420 0x7f6404153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f640a140640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64040d1eb0 0x7f64040cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f640a140640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f640415f430 0x7f64040c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f640a140640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f64040d01e0 con 0x7f64040b5420 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f640a140640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f64040cff00 con 0x7f64040d1eb0 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f640a140640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f64040d0e10 con 0x7f640415f430 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f64037fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64040b5420 0x7f6404153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f64037fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64040b5420 0x7f6404153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37018/0 (socket says 172.21.15.52:37018) 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f64037fe640 1 -- 172.21.15.52:0/1698937879 learned_addr learned my addr 172.21.15.52:0/1698937879 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f6402ffd640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64040d1eb0 0x7f64040cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f6403fff640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f640415f430 0x7f64040c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f64037fe640 1 -- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f640415f430 msgr2=0x7f64040c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f64037fe640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f640415f430 0x7f64040c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f64037fe640 1 -- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64040d1eb0 msgr2=0x7f64040cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f64037fe640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64040d1eb0 0x7f64040cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f64037fe640 1 -- 172.21.15.52:0/1698937879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64040cbae0 con 0x7f64040b5420 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f6402ffd640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64040d1eb0 0x7f64040cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.069+0000 7f64037fe640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64040b5420 0x7f6404153610 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000c00 tx=0x7f63f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.070+0000 7f63ebfff640 1 -- 172.21.15.52:0/1698937879 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4041070 con 0x7f64040b5420 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.070+0000 7f63ebfff640 1 -- 172.21.15.52:0/1698937879 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f403c870 con 0x7f64040b5420 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.070+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6404162260 con 0x7f64040b5420 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.070+0000 7f63ebfff640 1 -- 172.21.15.52:0/1698937879 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4037040 con 0x7f64040b5420 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.070+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64040ca7e0 con 0x7f64040b5420 2024-05-11T01:02:02.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.071+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64040695d0 con 0x7f64040b5420 2024-05-11T01:02:02.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.071+0000 7f63ebfff640 1 -- 172.21.15.52:0/1698937879 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7f63f4044070 con 0x7f64040b5420 2024-05-11T01:02:02.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.072+0000 7f63ebfff640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f63f0076560 0x7f63f0078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.072+0000 7f63ebfff640 1 -- 172.21.15.52:0/1698937879 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f63f40b87c0 con 0x7f64040b5420 2024-05-11T01:02:02.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.072+0000 7f6402ffd640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f63f0076560 0x7f63f0078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.073+0000 7f6402ffd640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f63f0076560 0x7f63f0078a20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006fd0 tx=0x7f63f8008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:02.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.076+0000 7f63ebfff640 1 -- 172.21.15.52:0/1698937879 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f63f4081cb0 con 0x7f64040b5420 2024-05-11T01:02:02.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.289+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f64040b5200 con 0x7f63f0076560 2024-05-11T01:02:02.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.346+0000 7f63ebfff640 1 -- 172.21.15.52:0/1698937879 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f64040b5200 con 0x7f63f0076560 2024-05-11T01:02:02.347 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir7 2024-05-11T01:02:02.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f63f0076560 msgr2=0x7f63f0078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f63f0076560 0x7f63f0078a20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006fd0 tx=0x7f63f8008040 comp rx=0 tx=0).stop 2024-05-11T01:02:02.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64040b5420 msgr2=0x7f6404153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64040b5420 0x7f6404153610 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000c00 tx=0x7f63f40046c0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 shutdown_connections 2024-05-11T01:02:02.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f640415f430 0x7f64040c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64040d1eb0 0x7f64040cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f63f0076560 0x7f63f0078a20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 --2- 172.21.15.52:0/1698937879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64040b5420 0x7f6404153610 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 >> 172.21.15.52:0/1698937879 conn(0x7f640408ebb0 msgr2=0x7f64040b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:02.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 shutdown_connections 2024-05-11T01:02:02.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.349+0000 7f640a140640 1 -- 172.21.15.52:0/1698937879 wait complete. 2024-05-11T01:02:02.360 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2024-05-11T01:02:02.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:02:02.360 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2024-05-11T01:02:02.381 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1m 2024-05-11T01:02:02.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.527+0000 7efe44919640 1 Processor -- start 2024-05-11T01:02:02.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.527+0000 7efe44919640 1 -- start start 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.528+0000 7efe44919640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 0x7efe4015fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.528+0000 7efe44919640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efe4006c480 0x7efe4006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.528+0000 7efe44919640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe4015c4b0 con 0x7efe4015c0d0 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.528+0000 7efe44919640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7efe4006ce10 con 0x7efe4006c480 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.528+0000 7efe44919640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efe40160ad0 con 0x7efe4006e880 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.528+0000 7efe3ed9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efe4006c480 0x7efe4006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.528+0000 7efe3ed9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efe4006c480 0x7efe4006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37026/0 (socket says 172.21.15.52:37026) 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.528+0000 7efe3ed9d640 1 -- 172.21.15.52:0/2487767673 learned_addr learned my addr 172.21.15.52:0/2487767673 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.529+0000 7efe3dd9b640 1 --2- 172.21.15.52:0/2487767673 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 0x7efe4015fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.529+0000 7efe3ed9d640 1 -- 172.21.15.52:0/2487767673 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 msgr2=0x7efe4015fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.529+0000 7efe3ed9d640 1 --2- 172.21.15.52:0/2487767673 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 0x7efe4015fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.529+0000 7efe3ed9d640 1 -- 172.21.15.52:0/2487767673 >> v1:172.21.15.119:6789/0 conn(0x7efe4015c0d0 legacy=0x7efe40072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.529+0000 7efe3ed9d640 1 -- 172.21.15.52:0/2487767673 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe40160510 con 0x7efe4006c480 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.529+0000 7efe3ed9d640 1 --2- 172.21.15.52:0/2487767673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4006c480 0x7efe4006c8d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efe38009870 tx=0x7efe3802a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e947b6ed2475a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe3d59a640 1 -- 172.21.15.52:0/2487767673 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38037070 con 0x7efe4006c480 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe3d59a640 1 -- 172.21.15.52:0/2487767673 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe38032440 con 0x7efe4006c480 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe3d59a640 1 -- 172.21.15.52:0/2487767673 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38032810 con 0x7efe4006c480 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe44919640 1 -- 172.21.15.52:0/2487767673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4006c480 msgr2=0x7efe4006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe44919640 1 --2- 172.21.15.52:0/2487767673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4006c480 0x7efe4006c8d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efe38009870 tx=0x7efe3802a1b0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe44919640 1 -- 172.21.15.52:0/2487767673 shutdown_connections 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe44919640 1 --2- 172.21.15.52:0/2487767673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4006c480 0x7efe4006c8d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe44919640 1 --2- 172.21.15.52:0/2487767673 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 0x7efe4015fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe44919640 1 -- 172.21.15.52:0/2487767673 >> 172.21.15.52:0/2487767673 conn(0x7efe400cd830 msgr2=0x7efe400cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:02.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe44919640 1 -- 172.21.15.52:0/2487767673 shutdown_connections 2024-05-11T01:02:02.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.530+0000 7efe44919640 1 -- 172.21.15.52:0/2487767673 wait complete. 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.532+0000 7efe44919640 1 Processor -- start 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.532+0000 7efe44919640 1 -- start start 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.532+0000 7efe44919640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4006c480 0x7efe400add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.532+0000 7efe44919640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 0x7efe400af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.532+0000 7efe44919640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4015c0d0 0x7efe4015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe44919640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe400cc490 con 0x7efe4015c0d0 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe44919640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe400cc1b0 con 0x7efe4006c480 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe44919640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efe401600e0 con 0x7efe4006e880 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4015c0d0 0x7efe4015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4015c0d0 0x7efe4015aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37032/0 (socket says 172.21.15.52:37032) 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3ed9d640 1 -- 172.21.15.52:0/29781117 learned_addr learned my addr 172.21.15.52:0/29781117 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:02.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3e59c640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4006c480 0x7efe400add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3dd9b640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 0x7efe400af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3ed9d640 1 -- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 msgr2=0x7efe400af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3ed9d640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 0x7efe400af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3ed9d640 1 -- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4006c480 msgr2=0x7efe400add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3ed9d640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4006c480 0x7efe400add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3ed9d640 1 -- 172.21.15.52:0/29781117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe400ae5a0 con 0x7efe4015c0d0 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe3ed9d640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4015c0d0 0x7efe4015aa60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efe38009870 tx=0x7efe380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe2effd640 1 -- 172.21.15.52:0/29781117 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38041020 con 0x7efe4015c0d0 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe2effd640 1 -- 172.21.15.52:0/29781117 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe3803d640 con 0x7efe4015c0d0 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe2effd640 1 -- 172.21.15.52:0/29781117 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38037040 con 0x7efe4015c0d0 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe400ae880 con 0x7efe4015c0d0 2024-05-11T01:02:02.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.533+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe400aee40 con 0x7efe4015c0d0 2024-05-11T01:02:02.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.534+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe400695d0 con 0x7efe4015c0d0 2024-05-11T01:02:02.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.535+0000 7efe2effd640 1 -- 172.21.15.52:0/29781117 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7efe38044020 con 0x7efe4015c0d0 2024-05-11T01:02:02.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.536+0000 7efe2effd640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe20076590 0x7efe20078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:02.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.536+0000 7efe2effd640 1 -- 172.21.15.52:0/29781117 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7efe380b81a0 con 0x7efe4015c0d0 2024-05-11T01:02:02.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.536+0000 7efe3e59c640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe20076590 0x7efe20078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:02.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.537+0000 7efe3e59c640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe20076590 0x7efe20078a50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efe30001e20 tx=0x7efe30009040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:02.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.540+0000 7efe2effd640 1 -- 172.21.15.52:0/29781117 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efe3804b030 con 0x7efe4015c0d0 2024-05-11T01:02:02.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.758+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7efe400cd330 con 0x7efe20076590 2024-05-11T01:02:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.816+0000 7efe2effd640 1 -- 172.21.15.52:0/29781117 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7efe400cd330 con 0x7efe20076590 2024-05-11T01:02:02.816 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir8 2024-05-11T01:02:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe20076590 msgr2=0x7efe20078a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe20076590 0x7efe20078a50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efe30001e20 tx=0x7efe30009040 comp rx=0 tx=0).stop 2024-05-11T01:02:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4015c0d0 msgr2=0x7efe4015aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4015c0d0 0x7efe4015aa60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efe38009870 tx=0x7efe380046c0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 shutdown_connections 2024-05-11T01:02:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe20076590 0x7efe20078a50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe4015c0d0 0x7efe4015aa60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe4006e880 0x7efe400af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 --2- 172.21.15.52:0/29781117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4006c480 0x7efe400add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 >> 172.21.15.52:0/29781117 conn(0x7efe400cd830 msgr2=0x7efe40155240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 shutdown_connections 2024-05-11T01:02:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.818+0000 7efe44919640 1 -- 172.21.15.52:0/29781117 wait complete. 2024-05-11T01:02:02.829 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2024-05-11T01:02:02.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:02:02.829 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2024-05-11T01:02:02.850 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1m 2024-05-11T01:02:02.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.998+0000 7fefc9bda640 1 Processor -- start 2024-05-11T01:02:02.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.998+0000 7fefc9bda640 1 -- start start 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.999+0000 7fefc9bda640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fefc4161bd0 0x7fefc4168320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.999+0000 7fefc9bda640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc40b2d20 con 0x7fefc40b28d0 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.999+0000 7fefc9bda640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc40ce7a0 con 0x7fefc41607e0 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.999+0000 7fefc9bda640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fefc40d0b40 con 0x7fefc4161bd0 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.999+0000 7fefc2ffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fefc40b28d0 0x7fefc40d19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36856/0 (socket says 172.21.15.52:36856) 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.999+0000 7fefc2ffd640 1 -- 172.21.15.52:0/3405731971 learned_addr learned my addr 172.21.15.52:0/3405731971 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.999+0000 7fefc3fff640 1 --2- 172.21.15.52:0/3405731971 >> v2:172.21.15.119:3300/0 conn(0x7fefc4161bd0 0x7fefc4168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.999+0000 7fefc27fc640 1 -- 172.21.15.52:0/3405731971 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 462476500 0 0) 0x7fefc40b2d20 con 0x7fefc40b28d0 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:02.999+0000 7fefc27fc640 1 -- 172.21.15.52:0/3405731971 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefac003520 con 0x7fefc40b28d0 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.000+0000 7fefc3fff640 1 -- 172.21.15.52:0/3405731971 >> v1:172.21.15.119:6789/0 conn(0x7fefc41607e0 legacy=0x7fefc40d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.000+0000 7fefc27fc640 1 -- 172.21.15.52:0/3405731971 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410181181 0 0) 0x7fefc40ce7a0 con 0x7fefc41607e0 2024-05-11T01:02:03.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.000+0000 7fefc3fff640 1 -- 172.21.15.52:0/3405731971 >> v1:172.21.15.52:6789/0 conn(0x7fefc40b28d0 legacy=0x7fefc40d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.000+0000 7fefc3fff640 1 -- 172.21.15.52:0/3405731971 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc40cea80 con 0x7fefc4161bd0 2024-05-11T01:02:03.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.000+0000 7fefc3fff640 1 --2- 172.21.15.52:0/3405731971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc4161bd0 0x7fefc4168320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009d90 tx=0x7fefbc02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c37e0555ff2be5e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:03.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc27fc640 1 -- 172.21.15.52:0/3405731971 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc037070 con 0x7fefc4161bd0 2024-05-11T01:02:03.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc27fc640 1 -- 172.21.15.52:0/3405731971 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc007740 con 0x7fefc4161bd0 2024-05-11T01:02:03.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc27fc640 1 -- 172.21.15.52:0/3405731971 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc007af0 con 0x7fefc4161bd0 2024-05-11T01:02:03.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc9bda640 1 -- 172.21.15.52:0/3405731971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc4161bd0 msgr2=0x7fefc4168320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc9bda640 1 --2- 172.21.15.52:0/3405731971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc4161bd0 0x7fefc4168320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009d90 tx=0x7fefbc02a780 comp rx=0 tx=0).stop 2024-05-11T01:02:03.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc9bda640 1 -- 172.21.15.52:0/3405731971 shutdown_connections 2024-05-11T01:02:03.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc9bda640 1 --2- 172.21.15.52:0/3405731971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc4161bd0 0x7fefc4168320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc9bda640 1 -- 172.21.15.52:0/3405731971 >> 172.21.15.52:0/3405731971 conn(0x7fefc40b4a60 msgr2=0x7fefc40b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:03.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc9bda640 1 -- 172.21.15.52:0/3405731971 shutdown_connections 2024-05-11T01:02:03.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.001+0000 7fefc9bda640 1 -- 172.21.15.52:0/3405731971 wait complete. 2024-05-11T01:02:03.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.002+0000 7fefc9bda640 1 Processor -- start 2024-05-11T01:02:03.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.002+0000 7fefc9bda640 1 -- start start 2024-05-11T01:02:03.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc9bda640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc40b28d0 0x7fefc4157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc9bda640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc41607e0 0x7fefc40b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc9bda640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc4161bd0 0x7fefc40b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc9bda640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40c9d00 con 0x7fefc41607e0 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc9bda640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40d0d90 con 0x7fefc40b28d0 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc9bda640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fefc40c9e80 con 0x7fefc4161bd0 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc41607e0 0x7fefc40b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc41607e0 0x7fefc40b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37038/0 (socket says 172.21.15.52:37038) 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc2ffd640 1 -- 172.21.15.52:0/1602387700 learned_addr learned my addr 172.21.15.52:0/1602387700 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc3fff640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc4161bd0 0x7fefc40b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.003+0000 7fefc37fe640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc40b28d0 0x7fefc4157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefc2ffd640 1 -- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc4161bd0 msgr2=0x7fefc40b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefc2ffd640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc4161bd0 0x7fefc40b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefc2ffd640 1 -- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc40b28d0 msgr2=0x7fefc4157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefc2ffd640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc40b28d0 0x7fefc4157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefc2ffd640 1 -- 172.21.15.52:0/1602387700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc40b84f0 con 0x7fefc41607e0 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefc37fe640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc40b28d0 0x7fefc4157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefc2ffd640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc41607e0 0x7fefc40b7980 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fefb800c500 tx=0x7fefb800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefabfff640 1 -- 172.21.15.52:0/1602387700 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8017070 con 0x7fefc41607e0 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefabfff640 1 -- 172.21.15.52:0/1602387700 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb8005ce0 con 0x7fefc41607e0 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc415df00 con 0x7fefc41607e0 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefabfff640 1 -- 172.21.15.52:0/1602387700 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8007270 con 0x7fefc41607e0 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.004+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc415e4c0 con 0x7fefc41607e0 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.005+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc40695d0 con 0x7fefc41607e0 2024-05-11T01:02:03.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.006+0000 7fefabfff640 1 -- 172.21.15.52:0/1602387700 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7fefb801a070 con 0x7fefc41607e0 2024-05-11T01:02:03.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.006+0000 7fefabfff640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefb0076560 0x7fefb0078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.006+0000 7fefabfff640 1 -- 172.21.15.52:0/1602387700 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7fefb8096e70 con 0x7fefc41607e0 2024-05-11T01:02:03.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.007+0000 7fefc37fe640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefb0076560 0x7fefb0078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.007+0000 7fefc37fe640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefb0076560 0x7fefb0078a20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000f60 tx=0x7fefb4009040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:03.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.011+0000 7fefabfff640 1 -- 172.21.15.52:0/1602387700 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fefb805fc90 con 0x7fefc41607e0 2024-05-11T01:02:03.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.231+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fefc40b87d0 con 0x7fefb0076560 2024-05-11T01:02:03.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.295+0000 7fefabfff640 1 -- 172.21.15.52:0/1602387700 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fefc40b87d0 con 0x7fefb0076560 2024-05-11T01:02:03.295 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path /snap_test_dir1/dir9 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefb0076560 msgr2=0x7fefb0078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefb0076560 0x7fefb0078a20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000f60 tx=0x7fefb4009040 comp rx=0 tx=0).stop 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc41607e0 msgr2=0x7fefc40b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc41607e0 0x7fefc40b7980 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fefb800c500 tx=0x7fefb800c9c0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 shutdown_connections 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fefc4161bd0 0x7fefc40b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefb0076560 0x7fefb0078a20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc41607e0 0x7fefc40b7980 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 --2- 172.21.15.52:0/1602387700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc40b28d0 0x7fefc4157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.297+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 >> 172.21.15.52:0/1602387700 conn(0x7fefc40b4a60 msgr2=0x7fefc409ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.298+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 shutdown_connections 2024-05-11T01:02:03.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.298+0000 7fefc9bda640 1 -- 172.21.15.52:0/1602387700 wait complete. 2024-05-11T01:02:03.308 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2024-05-11T01:02:03.309 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1m 2024-05-11T01:02:03.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.416+0000 7f8a1b6b4640 1 Processor -- start 2024-05-11T01:02:03.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.416+0000 7f8a1b6b4640 1 -- start start 2024-05-11T01:02:03.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a1b6b4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8a1415f4d0 0x7f8a140d50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a1b6b4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a140b5ee0 0x7f8a140d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a1b6b4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a140d3710 0x7f8a140d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a1b6b4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8a1415f8b0 con 0x7f8a1415f4d0 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a1b6b4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8a140b4e40 con 0x7f8a140d3710 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a1b6b4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a140b5120 con 0x7f8a140b5ee0 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19450640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8a1415f4d0 0x7f8a140d50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19450640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8a1415f4d0 0x7f8a140d50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37046/0 (socket says 172.21.15.52:37046) 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19450640 1 -- 172.21.15.52:0/639858820 learned_addr learned my addr 172.21.15.52:0/639858820 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a18c4f640 1 --2- 172.21.15.52:0/639858820 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a140b5ee0 0x7f8a140d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19c51640 1 --2- 172.21.15.52:0/639858820 >> v2:172.21.15.119:3300/0 conn(0x7f8a140d3710 0x7f8a140d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19450640 1 -- 172.21.15.52:0/639858820 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a140b5ee0 msgr2=0x7f8a140d5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19450640 1 --2- 172.21.15.52:0/639858820 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a140b5ee0 0x7f8a140d5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19450640 1 -- 172.21.15.52:0/639858820 >> v2:172.21.15.119:3300/0 conn(0x7f8a140d3710 msgr2=0x7f8a140d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19450640 1 --2- 172.21.15.52:0/639858820 >> v2:172.21.15.119:3300/0 conn(0x7f8a140d3710 0x7f8a140d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19450640 1 -- 172.21.15.52:0/639858820 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a140b5400 con 0x7f8a1415f4d0 2024-05-11T01:02:03.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a19c51640 1 --2- 172.21.15.52:0/639858820 >> v2:172.21.15.119:3300/0 conn(0x7f8a140d3710 0x7f8a140d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:02:03.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.417+0000 7f8a18c4f640 1 --2- 172.21.15.52:0/639858820 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a140b5ee0 0x7f8a140d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.418+0000 7f8a19450640 1 --2- 172.21.15.52:0/639858820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a1415f4d0 0x7f8a140d50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8a140d2b50 tx=0x7f8a1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63613a75fb06f64b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.418+0000 7f8a0bfff640 1 -- 172.21.15.52:0/639858820 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10037070 con 0x7f8a1415f4d0 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.418+0000 7f8a0bfff640 1 -- 172.21.15.52:0/639858820 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a10032440 con 0x7f8a1415f4d0 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.418+0000 7f8a0bfff640 1 -- 172.21.15.52:0/639858820 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10032810 con 0x7f8a1415f4d0 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.419+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/639858820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a1415f4d0 msgr2=0x7f8a140d50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.419+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/639858820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a1415f4d0 0x7f8a140d50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8a140d2b50 tx=0x7f8a1002a1e0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.419+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/639858820 shutdown_connections 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.419+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/639858820 >> v2:172.21.15.119:3300/0 conn(0x7f8a140d3710 0x7f8a140d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.419+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/639858820 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a140b5ee0 0x7f8a140d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.419+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/639858820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a1415f4d0 0x7f8a140d50c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.419+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/639858820 >> 172.21.15.52:0/639858820 conn(0x7f8a1408ebc0 msgr2=0x7f8a1409e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.419+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/639858820 shutdown_connections 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.419+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/639858820 wait complete. 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.420+0000 7f8a1b6b4640 1 Processor -- start 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.420+0000 7f8a1b6b4640 1 -- start start 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a1b6b4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140b5ee0 0x7f8a140a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a1b6b4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140d3710 0x7f8a1415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a1b6b4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f4d0 0x7f8a141e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a1b6b4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a140d40e0 con 0x7f8a140d3710 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a1b6b4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a140b59c0 con 0x7f8a140b5ee0 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a1b6b4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a140d1ab0 con 0x7f8a1415f4d0 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a18c4f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140d3710 0x7f8a1415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a18c4f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140d3710 0x7f8a1415db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37058/0 (socket says 172.21.15.52:37058) 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a18c4f640 1 -- 172.21.15.52:0/621440001 learned_addr learned my addr 172.21.15.52:0/621440001 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a19c51640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f4d0 0x7f8a141e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a19450640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140b5ee0 0x7f8a140a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a18c4f640 1 -- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f4d0 msgr2=0x7f8a141e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a18c4f640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f4d0 0x7f8a141e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a18c4f640 1 -- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140b5ee0 msgr2=0x7f8a140a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a18c4f640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140b5ee0 0x7f8a140a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a18c4f640 1 -- 172.21.15.52:0/621440001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a141e9110 con 0x7f8a140d3710 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a19c51640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f4d0 0x7f8a141e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a19450640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140b5ee0 0x7f8a140a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a18c4f640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140d3710 0x7f8a1415db10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c7c0 tx=0x7f8a0400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a09ffb640 1 -- 172.21.15.52:0/621440001 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04015070 con 0x7f8a140d3710 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a09ffb640 1 -- 172.21.15.52:0/621440001 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04007a30 con 0x7f8a140d3710 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.421+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a141e85a0 con 0x7f8a140d3710 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.422+0000 7f8a09ffb640 1 -- 172.21.15.52:0/621440001 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04007e00 con 0x7f8a140d3710 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.422+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1409ee50 con 0x7f8a140d3710 2024-05-11T01:02:03.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.423+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a140694f0 con 0x7f8a140d3710 2024-05-11T01:02:03.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.423+0000 7f8a09ffb640 1 -- 172.21.15.52:0/621440001 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7f8a04018070 con 0x7f8a140d3710 2024-05-11T01:02:03.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.424+0000 7f8a09ffb640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f89f8076560 0x7f89f8078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.424+0000 7f8a09ffb640 1 -- 172.21.15.52:0/621440001 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f8a04097b30 con 0x7f8a140d3710 2024-05-11T01:02:03.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.424+0000 7f8a19450640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f89f8076560 0x7f89f8078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.424+0000 7f8a19450640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f89f8076560 0x7f89f8078a20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8a10034000 tx=0x7f8a1002e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:03.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.428+0000 7f8a09ffb640 1 -- 172.21.15.52:0/621440001 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8a141e9110 con 0x7f8a140d3710 2024-05-11T01:02:03.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.645+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8a140d5440 con 0x7f89f8076560 2024-05-11T01:02:03.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.696+0000 7f8a09ffb640 1 -- 172.21.15.52:0/621440001 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8a140d5440 con 0x7f89f8076560 2024-05-11T01:02:03.696 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir0 2024-05-11T01:02:03.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f89f8076560 msgr2=0x7f89f8078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f89f8076560 0x7f89f8078a20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8a10034000 tx=0x7f8a1002e000 comp rx=0 tx=0).stop 2024-05-11T01:02:03.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140d3710 msgr2=0x7f8a1415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140d3710 0x7f8a1415db10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c7c0 tx=0x7f8a0400cc80 comp rx=0 tx=0).stop 2024-05-11T01:02:03.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 shutdown_connections 2024-05-11T01:02:03.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a1415f4d0 0x7f8a141e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f89f8076560 0x7f89f8078a20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140d3710 0x7f8a1415db10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 --2- 172.21.15.52:0/621440001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a140b5ee0 0x7f8a140a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 >> 172.21.15.52:0/621440001 conn(0x7f8a1408ebc0 msgr2=0x7f8a14161a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:03.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 shutdown_connections 2024-05-11T01:02:03.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.698+0000 7f8a1b6b4640 1 -- 172.21.15.52:0/621440001 wait complete. 2024-05-11T01:02:03.709 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2024-05-11T01:02:03.709 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1m 2024-05-11T01:02:03.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.815+0000 7ff1a76c8640 1 Processor -- start 2024-05-11T01:02:03.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.815+0000 7ff1a76c8640 1 -- start start 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.816+0000 7ff1a76c8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff1a015f4d0 0x7ff1a00d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.816+0000 7ff1a76c8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d3ef0 0x7ff1a00d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.816+0000 7ff1a76c8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a00d2e50 con 0x7ff1a00d1bd0 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.816+0000 7ff1a76c8640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a00d3130 con 0x7ff1a015f4d0 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.816+0000 7ff1a76c8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a00d3410 con 0x7ff1a00d3ef0 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.816+0000 7ff1a5464640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff1a015f4d0 0x7ff1a00d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.816+0000 7ff1a5464640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff1a015f4d0 0x7ff1a00d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37066/0 (socket says 172.21.15.52:37066) 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.816+0000 7ff1a5464640 1 -- 172.21.15.52:0/1738500873 learned_addr learned my addr 172.21.15.52:0/1738500873 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.817+0000 7ff1a4c63640 1 --2- 172.21.15.52:0/1738500873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d3ef0 0x7ff1a00d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.817+0000 7ff1a5464640 1 -- 172.21.15.52:0/1738500873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d3ef0 msgr2=0x7ff1a00d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.817+0000 7ff1a5464640 1 --2- 172.21.15.52:0/1738500873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d3ef0 0x7ff1a00d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.817+0000 7ff1a5464640 1 -- 172.21.15.52:0/1738500873 >> v1:172.21.15.119:6789/0 conn(0x7ff1a00d1bd0 legacy=0x7ff1a0163050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.817+0000 7ff1a5464640 1 -- 172.21.15.52:0/1738500873 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a00d36f0 con 0x7ff1a015f4d0 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.817+0000 7ff1a5464640 1 --2- 172.21.15.52:0/1738500873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 0x7ff1a00d07d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff190009e50 tx=0x7ff19002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce179ebba91617f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff19ffff640 1 -- 172.21.15.52:0/1738500873 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190036070 con 0x7ff1a015f4d0 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff19ffff640 1 -- 172.21.15.52:0/1738500873 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff190031600 con 0x7ff1a015f4d0 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff19ffff640 1 -- 172.21.15.52:0/1738500873 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1900319d0 con 0x7ff1a015f4d0 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff1a76c8640 1 -- 172.21.15.52:0/1738500873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 msgr2=0x7ff1a00d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff1a76c8640 1 --2- 172.21.15.52:0/1738500873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 0x7ff1a00d07d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff190009e50 tx=0x7ff19002a710 comp rx=0 tx=0).stop 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff1a76c8640 1 -- 172.21.15.52:0/1738500873 shutdown_connections 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff1a76c8640 1 --2- 172.21.15.52:0/1738500873 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d3ef0 0x7ff1a00d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff1a76c8640 1 --2- 172.21.15.52:0/1738500873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 0x7ff1a00d07d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff1a76c8640 1 -- 172.21.15.52:0/1738500873 >> 172.21.15.52:0/1738500873 conn(0x7ff1a009e500 msgr2=0x7ff1a00d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:03.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff1a76c8640 1 -- 172.21.15.52:0/1738500873 shutdown_connections 2024-05-11T01:02:03.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.818+0000 7ff1a76c8640 1 -- 172.21.15.52:0/1738500873 wait complete. 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.819+0000 7ff1a76c8640 1 Processor -- start 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.819+0000 7ff1a76c8640 1 -- start start 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a76c8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d1bd0 0x7ff1a015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a76c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a00d3ef0 0x7ff1a00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a76c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 0x7ff1a00b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a76c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a015f8b0 con 0x7ff1a015f4d0 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a76c8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a00d6090 con 0x7ff1a00d3ef0 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a76c8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a00d2060 con 0x7ff1a00d1bd0 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a5c65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 0x7ff1a00b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a5c65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 0x7ff1a00b7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37068/0 (socket says 172.21.15.52:37068) 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a5c65640 1 -- 172.21.15.52:0/2126597851 learned_addr learned my addr 172.21.15.52:0/2126597851 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a4c63640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a00d3ef0 0x7ff1a00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.820+0000 7ff1a5464640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d1bd0 0x7ff1a015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a5c65640 1 -- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d1bd0 msgr2=0x7ff1a015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a5c65640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d1bd0 0x7ff1a015fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a5c65640 1 -- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a00d3ef0 msgr2=0x7ff1a00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a5c65640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a00d3ef0 0x7ff1a00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a5c65640 1 -- 172.21.15.52:0/2126597851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a015ff70 con 0x7ff1a015f4d0 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a4c63640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a00d3ef0 0x7ff1a00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a5464640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d1bd0 0x7ff1a015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a5c65640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 0x7ff1a00b7380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c400 tx=0x7ff19800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff19dffb640 1 -- 172.21.15.52:0/2126597851 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198017070 con 0x7ff1a015f4d0 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff19dffb640 1 -- 172.21.15.52:0/2126597851 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1980052d0 con 0x7ff1a015f4d0 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a00b6000 con 0x7ff1a015f4d0 2024-05-11T01:02:03.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff19dffb640 1 -- 172.21.15.52:0/2126597851 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1980037f0 con 0x7ff1a015f4d0 2024-05-11T01:02:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.821+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a00b65c0 con 0x7ff1a015f4d0 2024-05-11T01:02:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.822+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a00694f0 con 0x7ff1a015f4d0 2024-05-11T01:02:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.823+0000 7ff19dffb640 1 -- 172.21.15.52:0/2126597851 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7ff19801a070 con 0x7ff1a015f4d0 2024-05-11T01:02:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.823+0000 7ff19dffb640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff18c076560 0x7ff18c078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.823+0000 7ff19dffb640 1 -- 172.21.15.52:0/2126597851 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7ff198096d90 con 0x7ff1a015f4d0 2024-05-11T01:02:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.824+0000 7ff1a5464640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff18c076560 0x7ff18c078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:03.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.824+0000 7ff1a5464640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff18c076560 0x7ff18c078a20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff190033000 tx=0x7ff190009000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:03.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:03.827+0000 7ff19dffb640 1 -- 172.21.15.52:0/2126597851 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff19805fd10 con 0x7ff1a015f4d0 2024-05-11T01:02:04.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.042+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff1a015ecd0 con 0x7ff18c076560 2024-05-11T01:02:04.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.092+0000 7ff19dffb640 1 -- 172.21.15.52:0/2126597851 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff1a015ecd0 con 0x7ff18c076560 2024-05-11T01:02:04.092 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir1 2024-05-11T01:02:04.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff18c076560 msgr2=0x7ff18c078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff18c076560 0x7ff18c078a20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff190033000 tx=0x7ff190009000 comp rx=0 tx=0).stop 2024-05-11T01:02:04.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 msgr2=0x7ff1a00b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 0x7ff1a00b7380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c400 tx=0x7ff19800c8c0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 shutdown_connections 2024-05-11T01:02:04.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff18c076560 0x7ff18c078a20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a015f4d0 0x7ff1a00b7380 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1a00d3ef0 0x7ff1a00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 --2- 172.21.15.52:0/2126597851 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1a00d1bd0 0x7ff1a015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 >> 172.21.15.52:0/2126597851 conn(0x7ff1a009e500 msgr2=0x7ff1a00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:04.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 shutdown_connections 2024-05-11T01:02:04.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.094+0000 7ff1a76c8640 1 -- 172.21.15.52:0/2126597851 wait complete. 2024-05-11T01:02:04.105 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2024-05-11T01:02:04.105 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1m 2024-05-11T01:02:04.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.212+0000 7efe287e4640 1 Processor -- start 2024-05-11T01:02:04.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.212+0000 7efe287e4640 1 -- start start 2024-05-11T01:02:04.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.213+0000 7efe287e4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 0x7efe200d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.213+0000 7efe287e4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efe200d3d20 0x7efe200b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.213+0000 7efe287e4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe2015f8b0 con 0x7efe200d2e10 2024-05-11T01:02:04.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.213+0000 7efe287e4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7efe200b8920 con 0x7efe200d3d20 2024-05-11T01:02:04.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.213+0000 7efe287e4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efe200b8c00 con 0x7efe2015f4d0 2024-05-11T01:02:04.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.214+0000 7efe26d81640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efe200d3d20 0x7efe200b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.214+0000 7efe26d81640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efe200d3d20 0x7efe200b99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37084/0 (socket says 172.21.15.52:37084) 2024-05-11T01:02:04.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.214+0000 7efe26d81640 1 -- 172.21.15.52:0/694920830 learned_addr learned my addr 172.21.15.52:0/694920830 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:04.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.214+0000 7efe26580640 1 --2- 172.21.15.52:0/694920830 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 0x7efe200d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.214+0000 7efe26d81640 1 -- 172.21.15.52:0/694920830 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 msgr2=0x7efe200d5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:04.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.214+0000 7efe26d81640 1 --2- 172.21.15.52:0/694920830 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 0x7efe200d5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.214+0000 7efe26d81640 1 -- 172.21.15.52:0/694920830 >> v1:172.21.15.119:6789/0 conn(0x7efe200d2e10 legacy=0x7efe200d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:04.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.214+0000 7efe26d81640 1 -- 172.21.15.52:0/694920830 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe200b8ee0 con 0x7efe200d3d20 2024-05-11T01:02:04.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.214+0000 7efe26580640 1 --2- 172.21.15.52:0/694920830 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 0x7efe200d5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:02:04.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe26d81640 1 --2- 172.21.15.52:0/694920830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d3d20 0x7efe200b99c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efe18009870 tx=0x7efe1802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199e1e73514a43d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:04.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe2557e640 1 -- 172.21.15.52:0/694920830 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18037070 con 0x7efe200d3d20 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe2557e640 1 -- 172.21.15.52:0/694920830 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe18032440 con 0x7efe200d3d20 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe2557e640 1 -- 172.21.15.52:0/694920830 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18032810 con 0x7efe200d3d20 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe287e4640 1 -- 172.21.15.52:0/694920830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d3d20 msgr2=0x7efe200b99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe287e4640 1 --2- 172.21.15.52:0/694920830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d3d20 0x7efe200b99c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efe18009870 tx=0x7efe1802a1c0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe287e4640 1 -- 172.21.15.52:0/694920830 shutdown_connections 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe287e4640 1 --2- 172.21.15.52:0/694920830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d3d20 0x7efe200b99c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe287e4640 1 --2- 172.21.15.52:0/694920830 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 0x7efe200d5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe287e4640 1 -- 172.21.15.52:0/694920830 >> 172.21.15.52:0/694920830 conn(0x7efe2009e450 msgr2=0x7efe200d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.215+0000 7efe287e4640 1 -- 172.21.15.52:0/694920830 shutdown_connections 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.216+0000 7efe287e4640 1 -- 172.21.15.52:0/694920830 wait complete. 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.217+0000 7efe287e4640 1 Processor -- start 2024-05-11T01:02:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.217+0000 7efe287e4640 1 -- start start 2024-05-11T01:02:04.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.217+0000 7efe287e4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d2e10 0x7efe20152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.217+0000 7efe287e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe200d3d20 0x7efe200cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.217+0000 7efe287e4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 0x7efe20153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.217+0000 7efe287e4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe200d3270 con 0x7efe200d2e10 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.217+0000 7efe287e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe200b94a0 con 0x7efe200d3d20 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.217+0000 7efe287e4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efe200b9f00 con 0x7efe2015f4d0 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.217+0000 7efe26580640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d2e10 0x7efe20152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe26580640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d2e10 0x7efe20152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37100/0 (socket says 172.21.15.52:37100) 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe26580640 1 -- 172.21.15.52:0/4072884407 learned_addr learned my addr 172.21.15.52:0/4072884407 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe26d81640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 0x7efe20153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe25d7f640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe200d3d20 0x7efe200cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe26580640 1 -- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 msgr2=0x7efe20153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe26580640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 0x7efe20153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe26580640 1 -- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe200d3d20 msgr2=0x7efe200cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe26580640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe200d3d20 0x7efe200cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe26580640 1 -- 172.21.15.52:0/4072884407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe20160bd0 con 0x7efe200d2e10 2024-05-11T01:02:04.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe25d7f640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe200d3d20 0x7efe200cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:02:04.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe26580640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d2e10 0x7efe20152da0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe1000ce00 tx=0x7efe1000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:04.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe1effd640 1 -- 172.21.15.52:0/4072884407 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1000f890 con 0x7efe200d2e10 2024-05-11T01:02:04.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe1effd640 1 -- 172.21.15.52:0/4072884407 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1000fa30 con 0x7efe200d2e10 2024-05-11T01:02:04.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe20160eb0 con 0x7efe200d2e10 2024-05-11T01:02:04.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe1effd640 1 -- 172.21.15.52:0/4072884407 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1000fe00 con 0x7efe200d2e10 2024-05-11T01:02:04.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.218+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe20161470 con 0x7efe200d2e10 2024-05-11T01:02:04.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.219+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe200694f0 con 0x7efe200d2e10 2024-05-11T01:02:04.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.220+0000 7efe1effd640 1 -- 172.21.15.52:0/4072884407 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7efe10018400 con 0x7efe200d2e10 2024-05-11T01:02:04.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.220+0000 7efe1effd640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe0c076560 0x7efe0c078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.220+0000 7efe1effd640 1 -- 172.21.15.52:0/4072884407 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7efe10098a50 con 0x7efe200d2e10 2024-05-11T01:02:04.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.220+0000 7efe25d7f640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe0c076560 0x7efe0c078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.221+0000 7efe25d7f640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe0c076560 0x7efe0c078a20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe200cf430 tx=0x7efe14009040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:04.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.225+0000 7efe1effd640 1 -- 172.21.15.52:0/4072884407 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efe10014030 con 0x7efe200d2e10 2024-05-11T01:02:04.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.440+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7efe200b7bf0 con 0x7efe0c076560 2024-05-11T01:02:04.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.489+0000 7efe1effd640 1 -- 172.21.15.52:0/4072884407 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7efe200b7bf0 con 0x7efe0c076560 2024-05-11T01:02:04.489 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir2 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe0c076560 msgr2=0x7efe0c078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe0c076560 0x7efe0c078a20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe200cf430 tx=0x7efe14009040 comp rx=0 tx=0).stop 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d2e10 msgr2=0x7efe20152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d2e10 0x7efe20152da0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe1000ce00 tx=0x7efe1000a650 comp rx=0 tx=0).stop 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 shutdown_connections 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe2015f4d0 0x7efe20153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe200d3d20 0x7efe200cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7efe0c076560 0x7efe0c078a20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 --2- 172.21.15.52:0/4072884407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe200d2e10 0x7efe20152da0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 >> 172.21.15.52:0/4072884407 conn(0x7efe2009e450 msgr2=0x7efe200d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:04.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 shutdown_connections 2024-05-11T01:02:04.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.491+0000 7efe287e4640 1 -- 172.21.15.52:0/4072884407 wait complete. 2024-05-11T01:02:04.502 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2024-05-11T01:02:04.502 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1m 2024-05-11T01:02:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.610+0000 7f449516a640 1 Processor -- start 2024-05-11T01:02:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.610+0000 7f449516a640 1 -- start start 2024-05-11T01:02:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.610+0000 7f449516a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f449015cdd0 0x7f4490160180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.610+0000 7f449516a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f44900d35b0 0x7f449006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.610+0000 7f449516a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44900b2c50 con 0x7f44901606c0 2024-05-11T01:02:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.610+0000 7f449516a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f44900b2f30 con 0x7f44900d35b0 2024-05-11T01:02:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.610+0000 7f449516a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f44900b3210 con 0x7f449015cdd0 2024-05-11T01:02:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.611+0000 7f448f59e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f44900d35b0 0x7f449006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.611+0000 7f448f59e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f44900d35b0 0x7f449006d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37106/0 (socket says 172.21.15.52:37106) 2024-05-11T01:02:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.611+0000 7f448f59e640 1 -- 172.21.15.52:0/2914954852 learned_addr learned my addr 172.21.15.52:0/2914954852 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.611+0000 7f448ed9d640 1 --2- 172.21.15.52:0/2914954852 >> v2:172.21.15.119:3300/0 conn(0x7f449015cdd0 0x7f4490160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.611+0000 7f448f59e640 1 -- 172.21.15.52:0/2914954852 >> v2:172.21.15.119:3300/0 conn(0x7f449015cdd0 msgr2=0x7f4490160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.611+0000 7f448f59e640 1 --2- 172.21.15.52:0/2914954852 >> v2:172.21.15.119:3300/0 conn(0x7f449015cdd0 0x7f4490160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.611+0000 7f448f59e640 1 -- 172.21.15.52:0/2914954852 >> v1:172.21.15.119:6789/0 conn(0x7f44901606c0 legacy=0x7f44900d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.611+0000 7f448f59e640 1 -- 172.21.15.52:0/2914954852 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44900b34f0 con 0x7f44900d35b0 2024-05-11T01:02:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f448f59e640 1 --2- 172.21.15.52:0/2914954852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 0x7f449006d590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f449009c740 tx=0x7f448802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdefd17d1fd710 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f448dd9b640 1 -- 172.21.15.52:0/2914954852 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488036070 con 0x7f44900d35b0 2024-05-11T01:02:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f448dd9b640 1 -- 172.21.15.52:0/2914954852 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4488031600 con 0x7f44900d35b0 2024-05-11T01:02:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f448dd9b640 1 -- 172.21.15.52:0/2914954852 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44880319d0 con 0x7f44900d35b0 2024-05-11T01:02:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f449516a640 1 -- 172.21.15.52:0/2914954852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 msgr2=0x7f449006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f449516a640 1 --2- 172.21.15.52:0/2914954852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 0x7f449006d590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f449009c740 tx=0x7f448802a610 comp rx=0 tx=0).stop 2024-05-11T01:02:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f449516a640 1 -- 172.21.15.52:0/2914954852 shutdown_connections 2024-05-11T01:02:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f449516a640 1 --2- 172.21.15.52:0/2914954852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 0x7f449006d590 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f449516a640 1 --2- 172.21.15.52:0/2914954852 >> v2:172.21.15.119:3300/0 conn(0x7f449015cdd0 0x7f4490160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f449516a640 1 -- 172.21.15.52:0/2914954852 >> 172.21.15.52:0/2914954852 conn(0x7f449009ba20 msgr2=0x7f449009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:04.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f449516a640 1 -- 172.21.15.52:0/2914954852 shutdown_connections 2024-05-11T01:02:04.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.612+0000 7f449516a640 1 -- 172.21.15.52:0/2914954852 wait complete. 2024-05-11T01:02:04.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.614+0000 7f449516a640 1 Processor -- start 2024-05-11T01:02:04.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.614+0000 7f449516a640 1 -- start start 2024-05-11T01:02:04.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f449516a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 0x7f449014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f449516a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f449015cdd0 0x7f44900c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f449516a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f44901606c0 0x7f4490151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f449516a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f449015d330 con 0x7f44900d35b0 2024-05-11T01:02:04.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f449516a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f449015d1b0 con 0x7f449015cdd0 2024-05-11T01:02:04.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f449516a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f44900b3cf0 con 0x7f44901606c0 2024-05-11T01:02:04.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 0x7f449014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 0x7f449014f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37114/0 (socket says 172.21.15.52:37114) 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448ed9d640 1 -- 172.21.15.52:0/3855496716 learned_addr learned my addr 172.21.15.52:0/3855496716 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448f59e640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f44901606c0 0x7f4490151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448e59c640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f449015cdd0 0x7f44900c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448ed9d640 1 -- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f44901606c0 msgr2=0x7f4490151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448ed9d640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f44901606c0 0x7f4490151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448ed9d640 1 -- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f449015cdd0 msgr2=0x7f44900c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448ed9d640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f449015cdd0 0x7f44900c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448ed9d640 1 -- 172.21.15.52:0/3855496716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4490152600 con 0x7f44900d35b0 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f448ed9d640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 0x7f449014f3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f448000c820 tx=0x7f448000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.615+0000 7f44777fe640 1 -- 172.21.15.52:0/3855496716 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480015070 con 0x7f44900d35b0 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.616+0000 7f44777fe640 1 -- 172.21.15.52:0/3855496716 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4480004240 con 0x7f44900d35b0 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.616+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44901528e0 con 0x7f44900d35b0 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.616+0000 7f44777fe640 1 -- 172.21.15.52:0/3855496716 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448000ce10 con 0x7f44900d35b0 2024-05-11T01:02:04.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.616+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44900c6ec0 con 0x7f44900d35b0 2024-05-11T01:02:04.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.617+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44900b1c30 con 0x7f44900d35b0 2024-05-11T01:02:04.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.617+0000 7f44777fe640 1 -- 172.21.15.52:0/3855496716 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7f4480007830 con 0x7f44900d35b0 2024-05-11T01:02:04.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.618+0000 7f44777fe640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f447c076560 0x7f447c078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:04.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.618+0000 7f44777fe640 1 -- 172.21.15.52:0/3855496716 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f4480097c00 con 0x7f44900d35b0 2024-05-11T01:02:04.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.618+0000 7f448e59c640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f447c076560 0x7f447c078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:04.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.619+0000 7f448e59c640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f447c076560 0x7f447c078a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f449015cb30 tx=0x7f4484009060 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:04.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.622+0000 7f44777fe640 1 -- 172.21.15.52:0/3855496716 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f44900b1c30 con 0x7f44900d35b0 2024-05-11T01:02:04.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.837+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f44900694f0 con 0x7f447c076560 2024-05-11T01:02:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.887+0000 7f44777fe640 1 -- 172.21.15.52:0/3855496716 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f44900694f0 con 0x7f447c076560 2024-05-11T01:02:04.887 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir3 2024-05-11T01:02:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f447c076560 msgr2=0x7f447c078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f447c076560 0x7f447c078a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f449015cb30 tx=0x7f4484009060 comp rx=0 tx=0).stop 2024-05-11T01:02:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 msgr2=0x7f449014f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 0x7f449014f3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f448000c820 tx=0x7f448000cce0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 shutdown_connections 2024-05-11T01:02:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f44901606c0 0x7f4490151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f449015cdd0 0x7f44900c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f447c076560 0x7f447c078a20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 --2- 172.21.15.52:0/3855496716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44900d35b0 0x7f449014f3e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:04.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 >> 172.21.15.52:0/3855496716 conn(0x7f449009ba20 msgr2=0x7f4490071090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:04.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 shutdown_connections 2024-05-11T01:02:04.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:04.889+0000 7f449516a640 1 -- 172.21.15.52:0/3855496716 wait complete. 2024-05-11T01:02:04.900 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2024-05-11T01:02:04.900 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1m 2024-05-11T01:02:05.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.007+0000 7f2bfe7c1640 1 Processor -- start 2024-05-11T01:02:05.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.007+0000 7f2bfe7c1640 1 -- start start 2024-05-11T01:02:05.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bfe7c1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2bf815f4f0 0x7f2bf80b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bfe7c1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bf815f8d0 con 0x7f2bf80ce260 2024-05-11T01:02:05.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bfe7c1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bf806dad0 con 0x7f2bf806dfa0 2024-05-11T01:02:05.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bfe7c1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf80b2520 con 0x7f2bf815f4f0 2024-05-11T01:02:05.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bf7fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2bf815f4f0 0x7f2bf80b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bfcd5e640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2bf80ce260 0x7f2bf80ce6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36872/0 (socket says 172.21.15.52:36872) 2024-05-11T01:02:05.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bfcd5e640 1 -- 172.21.15.52:0/2007447936 learned_addr learned my addr 172.21.15.52:0/2007447936 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:02:05.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bf7fff640 1 -- 172.21.15.52:0/2007447936 >> v1:172.21.15.119:6789/0 conn(0x7f2bf806dfa0 legacy=0x7f2bf806e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:05.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bf7fff640 1 -- 172.21.15.52:0/2007447936 >> v1:172.21.15.52:6789/0 conn(0x7f2bf80ce260 legacy=0x7f2bf80ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bf6ffd640 1 -- 172.21.15.52:0/2007447936 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 993288594 0 0) 0x7f2bf815f8d0 con 0x7f2bf80ce260 2024-05-11T01:02:05.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.008+0000 7f2bf7fff640 1 -- 172.21.15.52:0/2007447936 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf80cd780 con 0x7f2bf815f4f0 2024-05-11T01:02:05.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.009+0000 7f2bf7fff640 1 --2- 172.21.15.52:0/2007447936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf815f4f0 0x7f2bf80b55a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2be80097a0 tx=0x7f2be802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10bf3d32b3d2cf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:05.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.009+0000 7f2bf6ffd640 1 -- 172.21.15.52:0/2007447936 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8037070 con 0x7f2bf815f4f0 2024-05-11T01:02:05.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.009+0000 7f2bf6ffd640 1 -- 172.21.15.52:0/2007447936 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be8032440 con 0x7f2bf815f4f0 2024-05-11T01:02:05.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.009+0000 7f2bf6ffd640 1 -- 172.21.15.52:0/2007447936 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8032810 con 0x7f2bf815f4f0 2024-05-11T01:02:05.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.009+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/2007447936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf815f4f0 msgr2=0x7f2bf80b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.009+0000 7f2bfe7c1640 1 --2- 172.21.15.52:0/2007447936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf815f4f0 0x7f2bf80b55a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2be80097a0 tx=0x7f2be802a230 comp rx=0 tx=0).stop 2024-05-11T01:02:05.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.010+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/2007447936 shutdown_connections 2024-05-11T01:02:05.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.010+0000 7f2bfe7c1640 1 --2- 172.21.15.52:0/2007447936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf815f4f0 0x7f2bf80b55a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.010+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/2007447936 >> 172.21.15.52:0/2007447936 conn(0x7f2bf80b4a80 msgr2=0x7f2bf80b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:05.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.010+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/2007447936 shutdown_connections 2024-05-11T01:02:05.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.010+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/2007447936 wait complete. 2024-05-11T01:02:05.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.011+0000 7f2bfe7c1640 1 Processor -- start 2024-05-11T01:02:05.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.011+0000 7f2bfe7c1640 1 -- start start 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bfe7c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bf806dfa0 0x7f2bf81555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bfe7c1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf80ce260 0x7f2bf80b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bfe7c1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2bf815f4f0 0x7f2bf80b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bfe7c1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf806d7a0 con 0x7f2bf80ce260 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bfe7c1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf80cdd40 con 0x7f2bf806dfa0 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bfe7c1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf80d23f0 con 0x7f2bf815f4f0 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf80ce260 0x7f2bf80b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf80ce260 0x7f2bf80b80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37138/0 (socket says 172.21.15.52:37138) 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf77fe640 1 -- 172.21.15.52:0/4208689571 learned_addr learned my addr 172.21.15.52:0/4208689571 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf7fff640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bf806dfa0 0x7f2bf81555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bfcd5e640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2bf815f4f0 0x7f2bf80b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf77fe640 1 -- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2bf815f4f0 msgr2=0x7f2bf80b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf77fe640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2bf815f4f0 0x7f2bf80b85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf77fe640 1 -- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bf806dfa0 msgr2=0x7f2bf81555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf77fe640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bf806dfa0 0x7f2bf81555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf77fe640 1 -- 172.21.15.52:0/4208689571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf8162390 con 0x7f2bf80ce260 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf7fff640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bf806dfa0 0x7f2bf81555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.012+0000 7f2bf77fe640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf80ce260 0x7f2bf80b80b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00c430 tx=0x7f2bec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.013+0000 7f2bf4ff9640 1 -- 172.21.15.52:0/4208689571 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec017070 con 0x7f2bf80ce260 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.013+0000 7f2bf4ff9640 1 -- 172.21.15.52:0/4208689571 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bec0052d0 con 0x7f2bf80ce260 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.013+0000 7f2bf4ff9640 1 -- 172.21.15.52:0/4208689571 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec003830 con 0x7f2bf80ce260 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.013+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf80c7490 con 0x7f2bf80ce260 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.013+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf80c7a50 con 0x7f2bf80ce260 2024-05-11T01:02:05.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.014+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf80694f0 con 0x7f2bf80ce260 2024-05-11T01:02:05.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.014+0000 7f2bf4ff9640 1 -- 172.21.15.52:0/4208689571 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7f2bec01a070 con 0x7f2bf80ce260 2024-05-11T01:02:05.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.015+0000 7f2bf4ff9640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2be4076560 0x7f2be4078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.015+0000 7f2bf4ff9640 1 -- 172.21.15.52:0/4208689571 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f2bec096e30 con 0x7f2bf80ce260 2024-05-11T01:02:05.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.015+0000 7f2bf7fff640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2be4076560 0x7f2be4078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.015+0000 7f2bf7fff640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2be4076560 0x7f2be4078a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2be8034000 tx=0x7f2be802e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:05.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.019+0000 7f2bf4ff9640 1 -- 172.21.15.52:0/4208689571 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2bec060410 con 0x7f2bf80ce260 2024-05-11T01:02:05.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.234+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f2bf80c9dd0 con 0x7f2be4076560 2024-05-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.283+0000 7f2bf4ff9640 1 -- 172.21.15.52:0/4208689571 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2bf80c9dd0 con 0x7f2be4076560 2024-05-11T01:02:05.283 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir4 2024-05-11T01:02:05.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2be4076560 msgr2=0x7f2be4078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2be4076560 0x7f2be4078a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2be8034000 tx=0x7f2be802e000 comp rx=0 tx=0).stop 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf80ce260 msgr2=0x7f2bf80b80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf80ce260 0x7f2bf80b80b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00c430 tx=0x7f2bec00c8f0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 shutdown_connections 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2bf815f4f0 0x7f2bf80b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2be4076560 0x7f2be4078a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bf80ce260 0x7f2bf80b80b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 --2- 172.21.15.52:0/4208689571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bf806dfa0 0x7f2bf81555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 >> 172.21.15.52:0/4208689571 conn(0x7f2bf80b4a80 msgr2=0x7f2bf80b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 shutdown_connections 2024-05-11T01:02:05.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.285+0000 7f2bfe7c1640 1 -- 172.21.15.52:0/4208689571 wait complete. 2024-05-11T01:02:05.296 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2024-05-11T01:02:05.296 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1m 2024-05-11T01:02:05.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.403+0000 7f7b73104640 1 Processor -- start 2024-05-11T01:02:05.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.403+0000 7f7b73104640 1 -- start start 2024-05-11T01:02:05.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b73104640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b6c15f3c0 0x7f7b6c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b73104640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6c073e50 con 0x7f7b6c06c9c0 2024-05-11T01:02:05.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b73104640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6c074130 con 0x7f7b6c073980 2024-05-11T01:02:05.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b73104640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7b6c06ce30 con 0x7f7b6c15f3c0 2024-05-11T01:02:05.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b63fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f7b6c06c9c0 0x7f7b6c073270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36880/0 (socket says 172.21.15.52:36880) 2024-05-11T01:02:05.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b63fff640 1 -- 172.21.15.52:0/191104941 learned_addr learned my addr 172.21.15.52:0/191104941 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b70ea0640 1 --2- 172.21.15.52:0/191104941 >> v2:172.21.15.119:3300/0 conn(0x7f7b6c15f3c0 0x7f7b6c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136638867 0 0) 0x7f7b6c073e50 con 0x7f7b6c06c9c0 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b54003520 con 0x7f7b6c06c9c0 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 319612024 0 0) 0x7f7b54003520 con 0x7f7b6c06c9c0 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 >> v2:172.21.15.119:3300/0 conn(0x7f7b6c15f3c0 msgr2=0x7f7b6c06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 --2- 172.21.15.52:0/191104941 >> v2:172.21.15.119:3300/0 conn(0x7f7b6c15f3c0 0x7f7b6c06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 >> v1:172.21.15.119:6789/0 conn(0x7f7b6c073980 legacy=0x7f7b6c0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c0cfca0 con 0x7f7b6c06c9c0 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f7b5c0016f0 con 0x7f7b6c06c9c0 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b70ea0640 1 --2- 172.21.15.52:0/191104941 >> v2:172.21.15.119:3300/0 conn(0x7f7b6c15f3c0 0x7f7b6c06c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 >> v1:172.21.15.52:6789/0 conn(0x7f7b6c06c9c0 legacy=0x7f7b6c073270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.404+0000 7f7b637fe640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b54003a50 0x7f7b54003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b637fe640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b637fe640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b540051c0 0x7f7b5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c06ce30 con 0x7f7b54004800 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5400c160 con 0x7f7b540051c0 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5400c360 con 0x7f7b54003a50 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b63fff640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b716a1640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b540051c0 0x7f7b5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b70ea0640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b54003a50 0x7f7b54003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b63fff640 1 -- 172.21.15.52:0/191104941 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b54003a50 msgr2=0x7f7b54003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b63fff640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b54003a50 0x7f7b54003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b63fff640 1 -- 172.21.15.52:0/191104941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b540051c0 msgr2=0x7f7b5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b63fff640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b540051c0 0x7f7b5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b716a1640 1 -- 172.21.15.52:0/191104941 reap_dead start 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.405+0000 7f7b63fff640 1 -- 172.21.15.52:0/191104941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b54003520 con 0x7f7b54004800 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b63fff640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b54004c80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c240 tx=0x7f7b54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1097f5208e98d15 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c007d70 con 0x7f7b54004800 2024-05-11T01:02:05.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c005af0 con 0x7f7b54004800 2024-05-11T01:02:05.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b637fe640 1 -- 172.21.15.52:0/191104941 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c007070 con 0x7f7b54004800 2024-05-11T01:02:05.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b73104640 1 -- 172.21.15.52:0/191104941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 msgr2=0x7f7b54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b73104640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b54004c80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c240 tx=0x7f7b54005830 comp rx=0 tx=0).stop 2024-05-11T01:02:05.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b73104640 1 -- 172.21.15.52:0/191104941 shutdown_connections 2024-05-11T01:02:05.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b73104640 1 --2- 172.21.15.52:0/191104941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b54004c80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b73104640 1 -- 172.21.15.52:0/191104941 >> 172.21.15.52:0/191104941 conn(0x7f7b6c0ce940 msgr2=0x7f7b6c0ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:05.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.406+0000 7f7b73104640 1 -- 172.21.15.52:0/191104941 shutdown_connections 2024-05-11T01:02:05.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.407+0000 7f7b73104640 1 -- 172.21.15.52:0/191104941 wait complete. 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.408+0000 7f7b73104640 1 Processor -- start 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.408+0000 7f7b73104640 1 -- start start 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b73104640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b6c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b73104640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0ca9f0 0x7f7b6c1f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b73104640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cae70 0x7f7b6c1f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b73104640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c0d02f0 con 0x7f7b54004800 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b73104640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c15f7a0 con 0x7f7b6c0ca9f0 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b73104640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c06c480 con 0x7f7b6c0cae70 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b70ea0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b6c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b70ea0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b6c0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37156/0 (socket says 172.21.15.52:37156) 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b70ea0640 1 -- 172.21.15.52:0/3735206797 learned_addr learned my addr 172.21.15.52:0/3735206797 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b716a1640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cae70 0x7f7b6c1f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b63fff640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0ca9f0 0x7f7b6c1f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b70ea0640 1 -- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cae70 msgr2=0x7f7b6c1f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b70ea0640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cae70 0x7f7b6c1f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b70ea0640 1 -- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0ca9f0 msgr2=0x7f7b6c1f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b70ea0640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0ca9f0 0x7f7b6c1f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b70ea0640 1 -- 172.21.15.52:0/3735206797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c0cb3d0 con 0x7f7b54004800 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b63fff640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0ca9f0 0x7f7b6c1f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b70ea0640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b6c0b2e30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7b68011cd0 tx=0x7f7b6800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b617fa640 1 -- 172.21.15.52:0/3735206797 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800fd20 con 0x7f7b54004800 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b617fa640 1 -- 172.21.15.52:0/3735206797 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b68004020 con 0x7f7b54004800 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c0cb6b0 con 0x7f7b54004800 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b617fa640 1 -- 172.21.15.52:0/3735206797 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68004410 con 0x7f7b54004800 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.409+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c0b3370 con 0x7f7b54004800 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.410+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c0694f0 con 0x7f7b54004800 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.411+0000 7f7b617fa640 1 -- 172.21.15.52:0/3735206797 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7f7b68012040 con 0x7f7b54004800 2024-05-11T01:02:05.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.412+0000 7f7b617fa640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7b58076560 0x7f7b58078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.412+0000 7f7b617fa640 1 -- 172.21.15.52:0/3735206797 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f7b680980a0 con 0x7f7b54004800 2024-05-11T01:02:05.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.412+0000 7f7b63fff640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7b58076560 0x7f7b58078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.412+0000 7f7b63fff640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7b58076560 0x7f7b58078a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c032000 tx=0x7f7b5c009030 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:05.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.416+0000 7f7b617fa640 1 -- 172.21.15.52:0/3735206797 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7b68019030 con 0x7f7b54004800 2024-05-11T01:02:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.632+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7b6c0cf570 con 0x7f7b58076560 2024-05-11T01:02:05.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.679+0000 7f7b617fa640 1 -- 172.21.15.52:0/3735206797 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7b6c0cf570 con 0x7f7b58076560 2024-05-11T01:02:05.679 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir5 2024-05-11T01:02:05.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7b58076560 msgr2=0x7f7b58078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7b58076560 0x7f7b58078a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c032000 tx=0x7f7b5c009030 comp rx=0 tx=0).stop 2024-05-11T01:02:05.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 msgr2=0x7f7b6c0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b6c0b2e30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7b68011cd0 tx=0x7f7b6800f610 comp rx=0 tx=0).stop 2024-05-11T01:02:05.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 shutdown_connections 2024-05-11T01:02:05.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7b6c0cae70 0x7f7b6c1f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0ca9f0 0x7f7b6c1f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7b58076560 0x7f7b58078a20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 --2- 172.21.15.52:0/3735206797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b54004800 0x7f7b6c0b2e30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 >> 172.21.15.52:0/3735206797 conn(0x7f7b6c0ce940 msgr2=0x7f7b6c15ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:05.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 shutdown_connections 2024-05-11T01:02:05.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.681+0000 7f7b73104640 1 -- 172.21.15.52:0/3735206797 wait complete. 2024-05-11T01:02:05.692 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2024-05-11T01:02:05.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1m 2024-05-11T01:02:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.799+0000 7fdbab4a2640 1 Processor -- start 2024-05-11T01:02:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.799+0000 7fdbab4a2640 1 -- start start 2024-05-11T01:02:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdbab4a2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdba415f4d0 0x7fdba40d50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdbab4a2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40b5ee0 0x7fdba40d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdbab4a2640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdba40d3710 0x7fdba40d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdbab4a2640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdba415f8b0 con 0x7fdba40d3710 2024-05-11T01:02:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdbab4a2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fdba40b4e40 con 0x7fdba415f4d0 2024-05-11T01:02:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdbab4a2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdba40b5120 con 0x7fdba40b5ee0 2024-05-11T01:02:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba9a3f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdba40d3710 0x7fdba40d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba9a3f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdba40d3710 0x7fdba40d3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37168/0 (socket says 172.21.15.52:37168) 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba9a3f640 1 -- 172.21.15.52:0/2351897075 learned_addr learned my addr 172.21.15.52:0/2351897075 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba8a3d640 1 --2- 172.21.15.52:0/2351897075 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40b5ee0 0x7fdba40d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba923e640 1 --2- 172.21.15.52:0/2351897075 >> v2:172.21.15.119:3300/0 conn(0x7fdba415f4d0 0x7fdba40d50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba9a3f640 1 -- 172.21.15.52:0/2351897075 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40b5ee0 msgr2=0x7fdba40d5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba9a3f640 1 --2- 172.21.15.52:0/2351897075 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40b5ee0 0x7fdba40d5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba9a3f640 1 -- 172.21.15.52:0/2351897075 >> v2:172.21.15.119:3300/0 conn(0x7fdba415f4d0 msgr2=0x7fdba40d50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba9a3f640 1 --2- 172.21.15.52:0/2351897075 >> v2:172.21.15.119:3300/0 conn(0x7fdba415f4d0 0x7fdba40d50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.800+0000 7fdba9a3f640 1 -- 172.21.15.52:0/2351897075 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba40b5400 con 0x7fdba40d3710 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.801+0000 7fdba9a3f640 1 --2- 172.21.15.52:0/2351897075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40d3710 0x7fdba40d3ba0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdba408e8d0 tx=0x7fdb9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaec3c90cb4b3660 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.801+0000 7fdb9bfff640 1 -- 172.21.15.52:0/2351897075 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c036070 con 0x7fdba40d3710 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.801+0000 7fdb9bfff640 1 -- 172.21.15.52:0/2351897075 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9c031600 con 0x7fdba40d3710 2024-05-11T01:02:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.801+0000 7fdb9bfff640 1 -- 172.21.15.52:0/2351897075 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c0319d0 con 0x7fdba40d3710 2024-05-11T01:02:05.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.802+0000 7fdbab4a2640 1 -- 172.21.15.52:0/2351897075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40d3710 msgr2=0x7fdba40d3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.802+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/2351897075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40d3710 0x7fdba40d3ba0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdba408e8d0 tx=0x7fdb9c02a6a0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.802+0000 7fdbab4a2640 1 -- 172.21.15.52:0/2351897075 shutdown_connections 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.802+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/2351897075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40d3710 0x7fdba40d3ba0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.802+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/2351897075 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40b5ee0 0x7fdba40d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.802+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/2351897075 >> v2:172.21.15.119:3300/0 conn(0x7fdba415f4d0 0x7fdba40d50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.802+0000 7fdbab4a2640 1 -- 172.21.15.52:0/2351897075 >> 172.21.15.52:0/2351897075 conn(0x7fdba408ebc0 msgr2=0x7fdba409e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.802+0000 7fdbab4a2640 1 -- 172.21.15.52:0/2351897075 shutdown_connections 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.802+0000 7fdbab4a2640 1 -- 172.21.15.52:0/2351897075 wait complete. 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.803+0000 7fdbab4a2640 1 Processor -- start 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.803+0000 7fdbab4a2640 1 -- start start 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdbab4a2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b5ee0 0x7fdba415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdbab4a2640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40d3710 0x7fdba40ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdbab4a2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba415f4d0 0x7fdba40cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdbab4a2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdba40d40e0 con 0x7fdba40b5ee0 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdbab4a2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdba40b59c0 con 0x7fdba415f4d0 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdbab4a2640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdba40d1ab0 con 0x7fdba40d3710 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba923e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b5ee0 0x7fdba415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba923e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b5ee0 0x7fdba415c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37176/0 (socket says 172.21.15.52:37176) 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba923e640 1 -- 172.21.15.52:0/3320990519 learned_addr learned my addr 172.21.15.52:0/3320990519 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba9a3f640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba415f4d0 0x7fdba40cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba8a3d640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40d3710 0x7fdba40ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba923e640 1 -- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40d3710 msgr2=0x7fdba40ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba923e640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40d3710 0x7fdba40ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba923e640 1 -- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba415f4d0 msgr2=0x7fdba40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba923e640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba415f4d0 0x7fdba40cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba923e640 1 -- 172.21.15.52:0/3320990519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba415c5c0 con 0x7fdba40b5ee0 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba8a3d640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40d3710 0x7fdba40ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:02:05.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdba923e640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b5ee0 0x7fdba415c080 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e870 tx=0x7fdba000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.804+0000 7fdb99ffb640 1 -- 172.21.15.52:0/3320990519 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0017070 con 0x7fdba40b5ee0 2024-05-11T01:02:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.805+0000 7fdb99ffb640 1 -- 172.21.15.52:0/3320990519 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba000c940 con 0x7fdba40b5ee0 2024-05-11T01:02:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.805+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba40cd130 con 0x7fdba40b5ee0 2024-05-11T01:02:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.805+0000 7fdb99ffb640 1 -- 172.21.15.52:0/3320990519 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba000cd30 con 0x7fdba40b5ee0 2024-05-11T01:02:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.805+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba40cd6f0 con 0x7fdba40b5ee0 2024-05-11T01:02:05.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.806+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba40694f0 con 0x7fdba40b5ee0 2024-05-11T01:02:05.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.806+0000 7fdb99ffb640 1 -- 172.21.15.52:0/3320990519 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7fdba001a070 con 0x7fdba40b5ee0 2024-05-11T01:02:05.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.807+0000 7fdb99ffb640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdb88076560 0x7fdb88078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:05.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.807+0000 7fdb99ffb640 1 -- 172.21.15.52:0/3320990519 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7fdba0097920 con 0x7fdba40b5ee0 2024-05-11T01:02:05.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.807+0000 7fdba8a3d640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdb88076560 0x7fdb88078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:05.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.808+0000 7fdba8a3d640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdb88076560 0x7fdb88078a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdba40ce450 tx=0x7fdb94008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:05.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:05.811+0000 7fdb99ffb640 1 -- 172.21.15.52:0/3320990519 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdba0060860 con 0x7fdba40b5ee0 2024-05-11T01:02:06.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.029+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fdba40d5970 con 0x7fdb88076560 2024-05-11T01:02:06.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.079+0000 7fdb99ffb640 1 -- 172.21.15.52:0/3320990519 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fdba40d5970 con 0x7fdb88076560 2024-05-11T01:02:06.079 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir6 2024-05-11T01:02:06.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdb88076560 msgr2=0x7fdb88078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdb88076560 0x7fdb88078a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdba40ce450 tx=0x7fdb94008040 comp rx=0 tx=0).stop 2024-05-11T01:02:06.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b5ee0 msgr2=0x7fdba415c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b5ee0 0x7fdba415c080 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e870 tx=0x7fdba000ed30 comp rx=0 tx=0).stop 2024-05-11T01:02:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 shutdown_connections 2024-05-11T01:02:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba415f4d0 0x7fdba40cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdba40d3710 0x7fdba40ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdb88076560 0x7fdb88078a20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 --2- 172.21.15.52:0/3320990519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba40b5ee0 0x7fdba415c080 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 >> 172.21.15.52:0/3320990519 conn(0x7fdba408ebc0 msgr2=0x7fdba40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 shutdown_connections 2024-05-11T01:02:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.081+0000 7fdbab4a2640 1 -- 172.21.15.52:0/3320990519 wait complete. 2024-05-11T01:02:06.092 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2024-05-11T01:02:06.092 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1m 2024-05-11T01:02:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.201+0000 7f6fd3299640 1 Processor -- start 2024-05-11T01:02:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.201+0000 7f6fd3299640 1 -- start start 2024-05-11T01:02:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd3299640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc15f4d0 0x7f6fcc0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd3299640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6fcc0d3ef0 0x7f6fcc0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd3299640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fcc0d2e50 con 0x7f6fcc0d1bd0 2024-05-11T01:02:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd3299640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f6fcc0d3130 con 0x7f6fcc0d3ef0 2024-05-11T01:02:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd3299640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc0d3410 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd0834640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6fcc0d3ef0 0x7f6fcc0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd1836640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6fcc0d1bd0 0x7f6fcc163050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36890/0 (socket says 172.21.15.52:36890) 2024-05-11T01:02:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd0834640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6fcc0d3ef0 0x7f6fcc0d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37190/0 (socket says 172.21.15.52:37190) 2024-05-11T01:02:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd1836640 1 -- 172.21.15.52:0/126500097 learned_addr learned my addr 172.21.15.52:0/126500097 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd1035640 1 --2- 172.21.15.52:0/126500097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc15f4d0 0x7f6fcc0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd0834640 1 -- 172.21.15.52:0/126500097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc15f4d0 msgr2=0x7f6fcc0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd0834640 1 --2- 172.21.15.52:0/126500097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc15f4d0 0x7f6fcc0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd0834640 1 -- 172.21.15.52:0/126500097 >> v1:172.21.15.52:6789/0 conn(0x7f6fcc0d1bd0 legacy=0x7f6fcc163050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fd0834640 1 -- 172.21.15.52:0/126500097 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc0d36f0 con 0x7f6fcc0d3ef0 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.202+0000 7f6fbffff640 1 -- 172.21.15.52:0/126500097 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2198388673 0 0) 0x7f6fcc0d2e50 con 0x7f6fcc0d1bd0 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.203+0000 7f6fd0834640 1 --2- 172.21.15.52:0/126500097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc0d3ef0 0x7f6fcc0d0d10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80097a0 tx=0x7f6fc802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f3fc8fe0c51514 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.203+0000 7f6fbffff640 1 -- 172.21.15.52:0/126500097 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8037070 con 0x7f6fcc0d3ef0 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.203+0000 7f6fbffff640 1 -- 172.21.15.52:0/126500097 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc8032440 con 0x7f6fcc0d3ef0 2024-05-11T01:02:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.203+0000 7f6fbffff640 1 -- 172.21.15.52:0/126500097 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8032810 con 0x7f6fcc0d3ef0 2024-05-11T01:02:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.204+0000 7f6fd3299640 1 -- 172.21.15.52:0/126500097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc0d3ef0 msgr2=0x7f6fcc0d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.204+0000 7f6fd3299640 1 --2- 172.21.15.52:0/126500097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc0d3ef0 0x7f6fcc0d0d10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80097a0 tx=0x7f6fc802a180 comp rx=0 tx=0).stop 2024-05-11T01:02:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.204+0000 7f6fd3299640 1 -- 172.21.15.52:0/126500097 shutdown_connections 2024-05-11T01:02:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.204+0000 7f6fd3299640 1 --2- 172.21.15.52:0/126500097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc0d3ef0 0x7f6fcc0d0d10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.204+0000 7f6fd3299640 1 --2- 172.21.15.52:0/126500097 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc15f4d0 0x7f6fcc0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.204+0000 7f6fd3299640 1 -- 172.21.15.52:0/126500097 >> 172.21.15.52:0/126500097 conn(0x7f6fcc09e500 msgr2=0x7f6fcc0d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.204+0000 7f6fd3299640 1 -- 172.21.15.52:0/126500097 shutdown_connections 2024-05-11T01:02:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.204+0000 7f6fd3299640 1 -- 172.21.15.52:0/126500097 wait complete. 2024-05-11T01:02:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.205+0000 7f6fd3299640 1 Processor -- start 2024-05-11T01:02:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.205+0000 7f6fd3299640 1 -- start start 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd3299640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc0d1bd0 0x7f6fcc1f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd3299640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fcc0d3ef0 0x7f6fcc1f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd3299640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc15f4d0 0x7f6fcc0cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd3299640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc15f8b0 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd3299640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc0d6090 con 0x7f6fcc0d3ef0 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd3299640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc0d2060 con 0x7f6fcc0d1bd0 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1836640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc15f4d0 0x7f6fcc0cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1836640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc15f4d0 0x7f6fcc0cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37204/0 (socket says 172.21.15.52:37204) 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1836640 1 -- 172.21.15.52:0/1612631494 learned_addr learned my addr 172.21.15.52:0/1612631494 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1035640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc0d1bd0 0x7f6fcc1f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd0834640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fcc0d3ef0 0x7f6fcc1f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1836640 1 -- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc0d1bd0 msgr2=0x7f6fcc1f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1836640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc0d1bd0 0x7f6fcc1f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1836640 1 -- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fcc0d3ef0 msgr2=0x7f6fcc1f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1836640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fcc0d3ef0 0x7f6fcc1f83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1836640 1 -- 172.21.15.52:0/1612631494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc0cf600 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd0834640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fcc0d3ef0 0x7f6fcc1f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:02:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd1836640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc15f4d0 0x7f6fcc0cf0c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400c7a0 tx=0x7f6fc400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fbdffb640 1 -- 172.21.15.52:0/1612631494 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4018070 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fbdffb640 1 -- 172.21.15.52:0/1612631494 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc400a5e0 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.206+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc1f8ab0 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.207+0000 7f6fbdffb640 1 -- 172.21.15.52:0/1612631494 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400aa10 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.207+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc1f9070 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.208+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc1f92b0 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.208+0000 7f6fbdffb640 1 -- 172.21.15.52:0/1612631494 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7f6fc401b070 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.209+0000 7f6fbdffb640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6fb0076560 0x7f6fb0078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.209+0000 7f6fbdffb640 1 -- 172.21.15.52:0/1612631494 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f6fc4097110 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.209+0000 7f6fd1035640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6fb0076560 0x7f6fb0078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.209+0000 7f6fd1035640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6fb0076560 0x7f6fb0078a20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009aa0 tx=0x7f6fc0009190 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:06.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.213+0000 7f6fbdffb640 1 -- 172.21.15.52:0/1612631494 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6fcc1f92b0 con 0x7f6fcc15f4d0 2024-05-11T01:02:06.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.428+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f6fcc0694f0 con 0x7f6fb0076560 2024-05-11T01:02:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.476+0000 7f6fbdffb640 1 -- 172.21.15.52:0/1612631494 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6fcc0694f0 con 0x7f6fb0076560 2024-05-11T01:02:06.478 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir7 2024-05-11T01:02:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6fb0076560 msgr2=0x7f6fb0078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6fb0076560 0x7f6fb0078a20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009aa0 tx=0x7f6fc0009190 comp rx=0 tx=0).stop 2024-05-11T01:02:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc15f4d0 msgr2=0x7f6fcc0cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc15f4d0 0x7f6fcc0cf0c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400c7a0 tx=0x7f6fc400cc60 comp rx=0 tx=0).stop 2024-05-11T01:02:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 shutdown_connections 2024-05-11T01:02:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6fb0076560 0x7f6fb0078a20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fcc15f4d0 0x7f6fcc0cf0c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fcc0d3ef0 0x7f6fcc1f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 --2- 172.21.15.52:0/1612631494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fcc0d1bd0 0x7f6fcc1f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 >> 172.21.15.52:0/1612631494 conn(0x7f6fcc09e500 msgr2=0x7f6fcc0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:06.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 shutdown_connections 2024-05-11T01:02:06.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.478+0000 7f6fd3299640 1 -- 172.21.15.52:0/1612631494 wait complete. 2024-05-11T01:02:06.489 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2024-05-11T01:02:06.490 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1m 2024-05-11T01:02:06.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.599+0000 7f16c1d2b640 1 Processor -- start 2024-05-11T01:02:06.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.599+0000 7f16c1d2b640 1 -- start start 2024-05-11T01:02:06.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16c1d2b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f16bc15f4f0 0x7f16bc0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16c1d2b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16bc06dfa0 0x7f16bc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16c1d2b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16bc15f8d0 con 0x7f16bc0d1190 2024-05-11T01:02:06.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16c1d2b640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f16bc0b2520 con 0x7f16bc15f4f0 2024-05-11T01:02:06.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16c1d2b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc06dad0 con 0x7f16bc06dfa0 2024-05-11T01:02:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16bb7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f16bc15f4f0 0x7f16bc0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16bb7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f16bc15f4f0 0x7f16bc0b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37218/0 (socket says 172.21.15.52:37218) 2024-05-11T01:02:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16bb7fe640 1 -- 172.21.15.52:0/3355171661 learned_addr learned my addr 172.21.15.52:0/3355171661 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16bb7fe640 1 -- 172.21.15.52:0/3355171661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16bc06dfa0 msgr2=0x7f16bc06e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16bb7fe640 1 --2- 172.21.15.52:0/3355171661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16bc06dfa0 0x7f16bc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16bb7fe640 1 -- 172.21.15.52:0/3355171661 >> v1:172.21.15.119:6789/0 conn(0x7f16bc0d1190 legacy=0x7f16bc0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.600+0000 7f16bb7fe640 1 -- 172.21.15.52:0/3355171661 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc0cd780 con 0x7f16bc15f4f0 2024-05-11T01:02:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.601+0000 7f16bb7fe640 1 --2- 172.21.15.52:0/3355171661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc15f4f0 0x7f16bc0b55a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009870 tx=0x7f16ac02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fb538024645be9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.601+0000 7f16ba7fc640 1 -- 172.21.15.52:0/3355171661 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac037070 con 0x7f16bc15f4f0 2024-05-11T01:02:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.601+0000 7f16ba7fc640 1 -- 172.21.15.52:0/3355171661 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ac032440 con 0x7f16bc15f4f0 2024-05-11T01:02:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.601+0000 7f16ba7fc640 1 -- 172.21.15.52:0/3355171661 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac032810 con 0x7f16bc15f4f0 2024-05-11T01:02:06.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.602+0000 7f16c1d2b640 1 -- 172.21.15.52:0/3355171661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc15f4f0 msgr2=0x7f16bc0b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.602+0000 7f16c1d2b640 1 --2- 172.21.15.52:0/3355171661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc15f4f0 0x7f16bc0b55a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009870 tx=0x7f16ac02a1c0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.602+0000 7f16c1d2b640 1 -- 172.21.15.52:0/3355171661 shutdown_connections 2024-05-11T01:02:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.602+0000 7f16c1d2b640 1 --2- 172.21.15.52:0/3355171661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16bc06dfa0 0x7f16bc06e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.602+0000 7f16c1d2b640 1 --2- 172.21.15.52:0/3355171661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc15f4f0 0x7f16bc0b55a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.602+0000 7f16c1d2b640 1 -- 172.21.15.52:0/3355171661 >> 172.21.15.52:0/3355171661 conn(0x7f16bc0b4a80 msgr2=0x7f16bc0b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.602+0000 7f16c1d2b640 1 -- 172.21.15.52:0/3355171661 shutdown_connections 2024-05-11T01:02:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.602+0000 7f16c1d2b640 1 -- 172.21.15.52:0/3355171661 wait complete. 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.603+0000 7f16c1d2b640 1 Processor -- start 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.603+0000 7f16c1d2b640 1 -- start start 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16c1d2b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc06dfa0 0x7f16bc155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16c1d2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16bc0d1190 0x7f16bc161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16c1d2b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16bc15f4f0 0x7f16bc0b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16c1d2b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc06e900 con 0x7f16bc06dfa0 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16c1d2b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc0cdd40 con 0x7f16bc0d1190 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16c1d2b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc06d7a0 con 0x7f16bc15f4f0 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc06dfa0 0x7f16bc155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc06dfa0 0x7f16bc155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37234/0 (socket says 172.21.15.52:37234) 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bb7fe640 1 -- 172.21.15.52:0/30342949 learned_addr learned my addr 172.21.15.52:0/30342949 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16baffd640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16bc0d1190 0x7f16bc161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bbfff640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16bc15f4f0 0x7f16bc0b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bb7fe640 1 -- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16bc15f4f0 msgr2=0x7f16bc0b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bb7fe640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16bc15f4f0 0x7f16bc0b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bb7fe640 1 -- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16bc0d1190 msgr2=0x7f16bc161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bb7fe640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16bc0d1190 0x7f16bc161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bb7fe640 1 -- 172.21.15.52:0/30342949 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc160b30 con 0x7f16bc06dfa0 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.604+0000 7f16bb7fe640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc06dfa0 0x7f16bc155320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009870 tx=0x7f16ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.605+0000 7f16a3fff640 1 -- 172.21.15.52:0/30342949 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac041070 con 0x7f16bc06dfa0 2024-05-11T01:02:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.605+0000 7f16a3fff640 1 -- 172.21.15.52:0/30342949 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ac03c870 con 0x7f16bc06dfa0 2024-05-11T01:02:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.605+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc160e10 con 0x7f16bc06dfa0 2024-05-11T01:02:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.605+0000 7f16a3fff640 1 -- 172.21.15.52:0/30342949 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac037040 con 0x7f16bc06dfa0 2024-05-11T01:02:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.605+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc1613d0 con 0x7f16bc06dfa0 2024-05-11T01:02:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.606+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc0694f0 con 0x7f16bc06dfa0 2024-05-11T01:02:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.606+0000 7f16a3fff640 1 -- 172.21.15.52:0/30342949 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7f16ac044070 con 0x7f16bc06dfa0 2024-05-11T01:02:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.607+0000 7f16a3fff640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16a8076560 0x7f16a8078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.607+0000 7f16a3fff640 1 -- 172.21.15.52:0/30342949 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f16ac0b8670 con 0x7f16bc06dfa0 2024-05-11T01:02:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.607+0000 7f16baffd640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16a8076560 0x7f16a8078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:06.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.607+0000 7f16baffd640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16a8076560 0x7f16a8078a20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f16bc15e9d0 tx=0x7f16b0008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:06.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.611+0000 7f16a3fff640 1 -- 172.21.15.52:0/30342949 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f16ac081b70 con 0x7f16bc06dfa0 2024-05-11T01:02:06.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.827+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f16bc0b20d0 con 0x7f16a8076560 2024-05-11T01:02:06.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.874+0000 7f16a3fff640 1 -- 172.21.15.52:0/30342949 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f16bc0b20d0 con 0x7f16a8076560 2024-05-11T01:02:06.874 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir8 2024-05-11T01:02:06.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16a8076560 msgr2=0x7f16a8078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16a8076560 0x7f16a8078a20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f16bc15e9d0 tx=0x7f16b0008040 comp rx=0 tx=0).stop 2024-05-11T01:02:06.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc06dfa0 msgr2=0x7f16bc155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:06.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc06dfa0 0x7f16bc155320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009870 tx=0x7f16ac0046c0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 shutdown_connections 2024-05-11T01:02:06.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f16bc15f4f0 0x7f16bc0b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16bc0d1190 0x7f16bc161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16a8076560 0x7f16a8078a20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 --2- 172.21.15.52:0/30342949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16bc06dfa0 0x7f16bc155320 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:06.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 >> 172.21.15.52:0/30342949 conn(0x7f16bc0b4a80 msgr2=0x7f16bc0d1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:06.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 shutdown_connections 2024-05-11T01:02:06.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.876+0000 7f16c1d2b640 1 -- 172.21.15.52:0/30342949 wait complete. 2024-05-11T01:02:06.890 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2024-05-11T01:02:06.891 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1m 2024-05-11T01:02:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.997+0000 7f975736a640 1 Processor -- start 2024-05-11T01:02:06.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.998+0000 7f975736a640 1 -- start start 2024-05-11T01:02:07.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.998+0000 7f975736a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f97500d3d20 0x7f97500d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:07.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.998+0000 7f975736a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 0x7f97500b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:07.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.998+0000 7f975736a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f975015f8b0 con 0x7f975015f4d0 2024-05-11T01:02:07.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.998+0000 7f975736a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f97500b8920 con 0x7f97500d3d20 2024-05-11T01:02:07.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.998+0000 7f975736a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f97500b8c00 con 0x7f97500d2e10 2024-05-11T01:02:07.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.999+0000 7f9754905640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f97500d3d20 0x7f97500d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:07.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.999+0000 7f9754905640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f97500d3d20 0x7f97500d5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37248/0 (socket says 172.21.15.52:37248) 2024-05-11T01:02:07.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.999+0000 7f9754905640 1 -- 172.21.15.52:0/3121262532 learned_addr learned my addr 172.21.15.52:0/3121262532 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:07.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.999+0000 7f9755907640 1 --2- 172.21.15.52:0/3121262532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 0x7f97500b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.999+0000 7f9754905640 1 -- 172.21.15.52:0/3121262532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 msgr2=0x7f97500b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.999+0000 7f9754905640 1 --2- 172.21.15.52:0/3121262532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 0x7f97500b99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.999+0000 7f9754905640 1 -- 172.21.15.52:0/3121262532 >> v1:172.21.15.119:6789/0 conn(0x7f975015f4d0 legacy=0x7f97500d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:06.999+0000 7f9754905640 1 -- 172.21.15.52:0/3121262532 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97500b8ee0 con 0x7f97500d3d20 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f9754905640 1 --2- 172.21.15.52:0/3121262532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97500d3d20 0x7f97500d5a50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9740009870 tx=0x7f974002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b80f33cafc04d90 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f9747fff640 1 -- 172.21.15.52:0/3121262532 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740037070 con 0x7f97500d3d20 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f9747fff640 1 -- 172.21.15.52:0/3121262532 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9740032440 con 0x7f97500d3d20 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f9747fff640 1 -- 172.21.15.52:0/3121262532 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740032810 con 0x7f97500d3d20 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f975736a640 1 -- 172.21.15.52:0/3121262532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97500d3d20 msgr2=0x7f97500d5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f975736a640 1 --2- 172.21.15.52:0/3121262532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97500d3d20 0x7f97500d5a50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9740009870 tx=0x7f974002a270 comp rx=0 tx=0).stop 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f975736a640 1 -- 172.21.15.52:0/3121262532 shutdown_connections 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f975736a640 1 --2- 172.21.15.52:0/3121262532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 0x7f97500b99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f975736a640 1 --2- 172.21.15.52:0/3121262532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97500d3d20 0x7f97500d5a50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f975736a640 1 -- 172.21.15.52:0/3121262532 >> 172.21.15.52:0/3121262532 conn(0x7f975009e450 msgr2=0x7f97500d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f975736a640 1 -- 172.21.15.52:0/3121262532 shutdown_connections 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.000+0000 7f975736a640 1 -- 172.21.15.52:0/3121262532 wait complete. 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.002+0000 7f975736a640 1 Processor -- start 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.002+0000 7f975736a640 1 -- start start 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.002+0000 7f975736a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 0x7f9750150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.002+0000 7f975736a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97500d3d20 0x7f97500cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.002+0000 7f975736a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f975015f4d0 0x7f97501f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.002+0000 7f975736a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97500d3280 con 0x7f975015f4d0 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.002+0000 7f975736a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97500b94a0 con 0x7f97500d3d20 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.002+0000 7f975736a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f97500b9f00 con 0x7f97500d2e10 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.002+0000 7f9755907640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f975015f4d0 0x7f97501f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9755907640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f975015f4d0 0x7f97501f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37254/0 (socket says 172.21.15.52:37254) 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9755907640 1 -- 172.21.15.52:0/219693232 learned_addr learned my addr 172.21.15.52:0/219693232 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:02:07.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9754905640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97500d3d20 0x7f97500cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9755106640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 0x7f9750150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9755907640 1 -- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 msgr2=0x7f9750150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9755907640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 0x7f9750150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9755907640 1 -- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97500d3d20 msgr2=0x7f97500cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9755907640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97500d3d20 0x7f97500cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9755907640 1 -- 172.21.15.52:0/219693232 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97500c98e0 con 0x7f975015f4d0 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9755907640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f975015f4d0 0x7f97501f5410 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f974c007e40 tx=0x7f974c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9745ffb640 1 -- 172.21.15.52:0/219693232 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c014620 con 0x7f975015f4d0 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9745ffb640 1 -- 172.21.15.52:0/219693232 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974c019070 con 0x7f975015f4d0 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f9745ffb640 1 -- 172.21.15.52:0/219693232 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c014a20 con 0x7f975015f4d0 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97500c9b70 con 0x7f975015f4d0 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.003+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97500ca130 con 0x7f975015f4d0 2024-05-11T01:02:07.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.004+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97500694f0 con 0x7f975015f4d0 2024-05-11T01:02:07.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.005+0000 7f9745ffb640 1 -- 172.21.15.52:0/219693232 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 10) ==== 98434+0+0 (secure 0 0 0) 0x7f974c014bc0 con 0x7f975015f4d0 2024-05-11T01:02:07.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.005+0000 7f9745ffb640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9734076560 0x7f9734078a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:02:07.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.005+0000 7f9745ffb640 1 -- 172.21.15.52:0/219693232 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f974c098280 con 0x7f975015f4d0 2024-05-11T01:02:07.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.006+0000 7f9755106640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9734076560 0x7f9734078a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:02:07.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.006+0000 7f9755106640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9734076560 0x7f9734078a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9748001720 tx=0x7f9748009040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:02:07.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.010+0000 7f9745ffb640 1 -- 172.21.15.52:0/219693232 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f974c061200 con 0x7f975015f4d0 2024-05-11T01:02:07.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.225+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9750069e40 con 0x7f9734076560 2024-05-11T01:02:07.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.274+0000 7f9745ffb640 1 -- 172.21.15.52:0/219693232 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9750069e40 con 0x7f9734076560 2024-05-11T01:02:07.275 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule activated for path /snap_test_dir1/dir9 2024-05-11T01:02:07.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9734076560 msgr2=0x7f9734078a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:07.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9734076560 0x7f9734078a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9748001720 tx=0x7f9748009040 comp rx=0 tx=0).stop 2024-05-11T01:02:07.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f975015f4d0 msgr2=0x7f97501f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:02:07.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f975015f4d0 0x7f97501f5410 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f974c007e40 tx=0x7f974c00e4b0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 shutdown_connections 2024-05-11T01:02:07.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9734076560 0x7f9734078a20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f975015f4d0 0x7f97501f5410 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97500d3d20 0x7f97500cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 --2- 172.21.15.52:0/219693232 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97500d2e10 0x7f9750150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:02:07.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 >> 172.21.15.52:0/219693232 conn(0x7f975009e450 msgr2=0x7f9750161ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:02:07.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 shutdown_connections 2024-05-11T01:02:07.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:02:07.277+0000 7f975736a640 1 -- 172.21.15.52:0/219693232 wait complete. 2024-05-11T01:02:07.288 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2024-05-11T01:02:11.123 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:02:11.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:02:41.204 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:02:41.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:03:11.292 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:03:11.297 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:03:41.423 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:03:41.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:04:11.486 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:04:11.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:04:41.757 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:04:41.761 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:05:11.848 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:05:11.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:05:41.938 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:05:41.942 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:06:00.181 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:06:00.180+0000 7f89ccb44640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:06:00.181 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:06:00.180+0000 7f89ccb44640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:06:00.181 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:06:00.180+0000 7f89ccb44640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:06:00.181 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:06:00.180+0000 7f89ccb44640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:06:00.181 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:06:00.180+0000 7f89ccb44640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:06:12.004 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:06:12.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:06:42.135 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:06:42.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:07:12.281 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:07:12.285 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:07:42.428 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:07:42.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:08:12.575 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:08:12.578 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:08:42.695 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:08:42.699 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:09:12.783 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:09:12.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:09:42.969 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:09:42.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:10:13.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:10:13.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:10:43.122 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:10:43.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:11:13.245 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:11:13.249 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:11:43.316 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:11:43.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:12:13.446 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:12:13.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:12:43.512 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:12:43.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:13:13.657 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:13:13.661 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:13:19.288 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1m 2024-05-11T01:13:19.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.410+0000 7fdc9ef38640 1 Processor -- start 2024-05-11T01:13:19.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.410+0000 7fdc9ef38640 1 -- start start 2024-05-11T01:13:19.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.410+0000 7fdc9ef38640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdc980b3940 0x7fdc9806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc9ef38640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc98160d00 con 0x7fdc980b3d20 2024-05-11T01:13:19.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc9ef38640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc98160fc0 con 0x7fdc9815e0d0 2024-05-11T01:13:19.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc9ef38640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdc9806f8e0 con 0x7fdc980b3940 2024-05-11T01:13:19.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc9ccd4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdc980b3940 0x7fdc9806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc97fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fdc980b3d20 0x7fdc9815d9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:53414/0 (socket says 172.21.15.52:53414) 2024-05-11T01:13:19.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc9ccd4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdc980b3940 0x7fdc9806f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52176/0 (socket says 172.21.15.52:52176) 2024-05-11T01:13:19.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc97fff640 1 -- 172.21.15.52:0/1729414195 learned_addr learned my addr 172.21.15.52:0/1729414195 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:19.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc9ccd4640 1 -- 172.21.15.52:0/1729414195 >> v1:172.21.15.119:6789/0 conn(0x7fdc9815e0d0 legacy=0x7fdc980b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc9ccd4640 1 -- 172.21.15.52:0/1729414195 >> v1:172.21.15.52:6789/0 conn(0x7fdc980b3d20 legacy=0x7fdc9815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.411+0000 7fdc9ccd4640 1 -- 172.21.15.52:0/1729414195 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc981612a0 con 0x7fdc980b3940 2024-05-11T01:13:19.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.412+0000 7fdc9ccd4640 1 --2- 172.21.15.52:0/1729414195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3940 0x7fdc9806f3a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdc900097a0 tx=0x7fdc9002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6400f14166face server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:19.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.412+0000 7fdc977fe640 1 -- 172.21.15.52:0/1729414195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90037070 con 0x7fdc980b3940 2024-05-11T01:13:19.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.412+0000 7fdc977fe640 1 -- 172.21.15.52:0/1729414195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc90032440 con 0x7fdc980b3940 2024-05-11T01:13:19.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.412+0000 7fdc977fe640 1 -- 172.21.15.52:0/1729414195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90032810 con 0x7fdc980b3940 2024-05-11T01:13:19.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.413+0000 7fdc9ef38640 1 -- 172.21.15.52:0/1729414195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3940 msgr2=0x7fdc9806f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.413+0000 7fdc9ef38640 1 --2- 172.21.15.52:0/1729414195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3940 0x7fdc9806f3a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdc900097a0 tx=0x7fdc9002a230 comp rx=0 tx=0).stop 2024-05-11T01:13:19.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.413+0000 7fdc9ef38640 1 -- 172.21.15.52:0/1729414195 shutdown_connections 2024-05-11T01:13:19.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.413+0000 7fdc9ef38640 1 --2- 172.21.15.52:0/1729414195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3940 0x7fdc9806f3a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.413+0000 7fdc9ef38640 1 -- 172.21.15.52:0/1729414195 >> 172.21.15.52:0/1729414195 conn(0x7fdc98075ca0 msgr2=0x7fdc98076080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:19.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.413+0000 7fdc9ef38640 1 -- 172.21.15.52:0/1729414195 shutdown_connections 2024-05-11T01:13:19.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.413+0000 7fdc9ef38640 1 -- 172.21.15.52:0/1729414195 wait complete. 2024-05-11T01:13:19.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.414+0000 7fdc9ef38640 1 Processor -- start 2024-05-11T01:13:19.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.414+0000 7fdc9ef38640 1 -- start start 2024-05-11T01:13:19.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc9ef38640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc980b3940 0x7fdc981e7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc9ef38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3d20 0x7fdc98159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc9ef38640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9815e0d0 0x7fdc981f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc9ef38640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98161b40 con 0x7fdc980b3d20 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc9ef38640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98161860 con 0x7fdc980b3940 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc9ef38640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdc9815e560 con 0x7fdc9815e0d0 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3d20 0x7fdc98159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3d20 0x7fdc98159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52180/0 (socket says 172.21.15.52:52180) 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc97fff640 1 -- 172.21.15.52:0/3334610883 learned_addr learned my addr 172.21.15.52:0/3334610883 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc9ccd4640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc980b3940 0x7fdc981e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc9d4d5640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9815e0d0 0x7fdc981f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc97fff640 1 -- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9815e0d0 msgr2=0x7fdc981f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc97fff640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9815e0d0 0x7fdc981f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc97fff640 1 -- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc980b3940 msgr2=0x7fdc981e7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc97fff640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc980b3940 0x7fdc981e7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc97fff640 1 -- 172.21.15.52:0/3334610883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc981f33e0 con 0x7fdc980b3d20 2024-05-11T01:13:19.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.415+0000 7fdc9d4d5640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9815e0d0 0x7fdc981f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:19.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.416+0000 7fdc97fff640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3d20 0x7fdc98159340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800cba0 tx=0x7fdc8800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:19.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.416+0000 7fdc957fa640 1 -- 172.21.15.52:0/3334610883 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88017070 con 0x7fdc980b3d20 2024-05-11T01:13:19.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.416+0000 7fdc957fa640 1 -- 172.21.15.52:0/3334610883 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8800ad40 con 0x7fdc980b3d20 2024-05-11T01:13:19.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.416+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc981e7e10 con 0x7fdc980b3d20 2024-05-11T01:13:19.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.416+0000 7fdc957fa640 1 -- 172.21.15.52:0/3334610883 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88012600 con 0x7fdc980b3d20 2024-05-11T01:13:19.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.416+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc981f3650 con 0x7fdc980b3d20 2024-05-11T01:13:19.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.417+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc980695d0 con 0x7fdc980b3d20 2024-05-11T01:13:19.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.417+0000 7fdc957fa640 1 -- 172.21.15.52:0/3334610883 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fdc8801a070 con 0x7fdc980b3d20 2024-05-11T01:13:19.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.418+0000 7fdc957fa640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdc780764d0 0x7fdc78078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.418+0000 7fdc957fa640 1 -- 172.21.15.52:0/3334610883 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7fdc880976c0 con 0x7fdc980b3d20 2024-05-11T01:13:19.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.418+0000 7fdc9ccd4640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdc780764d0 0x7fdc78078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.419+0000 7fdc9ccd4640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdc780764d0 0x7fdc78078990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc90034000 tx=0x7fdc9002e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:19.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.423+0000 7fdc957fa640 1 -- 172.21.15.52:0/3334610883 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdc88060700 con 0x7fdc980b3d20 2024-05-11T01:13:19.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.638+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fdc98072af0 con 0x7fdc780764d0 2024-05-11T01:13:19.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.716+0000 7fdc957fa640 1 -- 172.21.15.52:0/3334610883 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fdc98072af0 con 0x7fdc780764d0 2024-05-11T01:13:19.716 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2024-05-11T01:13:19.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdc780764d0 msgr2=0x7fdc78078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdc780764d0 0x7fdc78078990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc90034000 tx=0x7fdc9002e000 comp rx=0 tx=0).stop 2024-05-11T01:13:19.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3d20 msgr2=0x7fdc98159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3d20 0x7fdc98159340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800cba0 tx=0x7fdc8800a430 comp rx=0 tx=0).stop 2024-05-11T01:13:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 shutdown_connections 2024-05-11T01:13:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9815e0d0 0x7fdc981f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fdc780764d0 0x7fdc78078990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc980b3d20 0x7fdc98159340 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 --2- 172.21.15.52:0/3334610883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc980b3940 0x7fdc981e7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 >> 172.21.15.52:0/3334610883 conn(0x7fdc98075ca0 msgr2=0x7fdc98142910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 shutdown_connections 2024-05-11T01:13:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.719+0000 7fdc9ef38640 1 -- 172.21.15.52:0/3334610883 wait complete. 2024-05-11T01:13:19.729 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2024-05-11T01:13:19.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1m 2024-05-11T01:13:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.837+0000 7f78f1674640 1 Processor -- start 2024-05-11T01:13:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.837+0000 7f78f1674640 1 -- start start 2024-05-11T01:13:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.837+0000 7f78f1674640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f78ec072330 0x7f78ec0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.837+0000 7f78f1674640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78ec06f700 con 0x7f78ec06c480 2024-05-11T01:13:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.837+0000 7f78f1674640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78ec06f9e0 con 0x7f78ec15d0e0 2024-05-11T01:13:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.837+0000 7f78f1674640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f78ec06ff80 con 0x7f78ec072330 2024-05-11T01:13:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.838+0000 7f78ea7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f78ec072330 0x7f78ec0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.838+0000 7f78eb7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f78ec06c480 0x7f78ec161070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:53424/0 (socket says 172.21.15.52:53424) 2024-05-11T01:13:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.838+0000 7f78eb7fe640 1 -- 172.21.15.52:0/1910100880 learned_addr learned my addr 172.21.15.52:0/1910100880 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.838+0000 7f78ea7fc640 1 -- 172.21.15.52:0/1910100880 >> v1:172.21.15.119:6789/0 conn(0x7f78ec15d0e0 legacy=0x7f78ec0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.838+0000 7f78ea7fc640 1 -- 172.21.15.52:0/1910100880 >> v1:172.21.15.52:6789/0 conn(0x7f78ec06c480 legacy=0x7f78ec161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.838+0000 7f78ea7fc640 1 -- 172.21.15.52:0/1910100880 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec070260 con 0x7f78ec072330 2024-05-11T01:13:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78ea7fc640 1 --2- 172.21.15.52:0/1910100880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec072330 0x7f78ec0b0200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f78e00097a0 tx=0x7f78e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7092d661ae51a6c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78e9ffb640 1 -- 172.21.15.52:0/1910100880 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0037070 con 0x7f78ec072330 2024-05-11T01:13:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78e9ffb640 1 -- 172.21.15.52:0/1910100880 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e0032440 con 0x7f78ec072330 2024-05-11T01:13:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78e9ffb640 1 -- 172.21.15.52:0/1910100880 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0032810 con 0x7f78ec072330 2024-05-11T01:13:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78f1674640 1 -- 172.21.15.52:0/1910100880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec072330 msgr2=0x7f78ec0b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78f1674640 1 --2- 172.21.15.52:0/1910100880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec072330 0x7f78ec0b0200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f78e00097a0 tx=0x7f78e002a230 comp rx=0 tx=0).stop 2024-05-11T01:13:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78f1674640 1 -- 172.21.15.52:0/1910100880 shutdown_connections 2024-05-11T01:13:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78f1674640 1 --2- 172.21.15.52:0/1910100880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec072330 0x7f78ec0b0200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78f1674640 1 -- 172.21.15.52:0/1910100880 >> 172.21.15.52:0/1910100880 conn(0x7f78ec0cd830 msgr2=0x7f78ec0cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78f1674640 1 -- 172.21.15.52:0/1910100880 shutdown_connections 2024-05-11T01:13:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.839+0000 7f78f1674640 1 -- 172.21.15.52:0/1910100880 wait complete. 2024-05-11T01:13:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.841+0000 7f78f1674640 1 Processor -- start 2024-05-11T01:13:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.841+0000 7f78f1674640 1 -- start start 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.841+0000 7f78f1674640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec06c480 0x7f78ec155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.841+0000 7f78f1674640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ec072330 0x7f78ec0c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.841+0000 7f78f1674640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78ec15d0e0 0x7f78ec0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.841+0000 7f78f1674640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec161900 con 0x7f78ec06c480 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.841+0000 7f78f1674640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec161780 con 0x7f78ec072330 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.841+0000 7f78f1674640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f78ec161a80 con 0x7f78ec15d0e0 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec06c480 0x7f78ec155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec06c480 0x7f78ec155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52190/0 (socket says 172.21.15.52:52190) 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eaffd640 1 -- 172.21.15.52:0/695322069 learned_addr learned my addr 172.21.15.52:0/695322069 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78ea7fc640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ec072330 0x7f78ec0c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eb7fe640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78ec15d0e0 0x7f78ec0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eaffd640 1 -- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78ec15d0e0 msgr2=0x7f78ec0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eaffd640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78ec15d0e0 0x7f78ec0c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eaffd640 1 -- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ec072330 msgr2=0x7f78ec0c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eaffd640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ec072330 0x7f78ec0c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eaffd640 1 -- 172.21.15.52:0/695322069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec09bf30 con 0x7f78ec06c480 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78ea7fc640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ec072330 0x7f78ec0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78eaffd640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec06c480 0x7f78ec155f70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c3d0 tx=0x7f78dc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78d37fe640 1 -- 172.21.15.52:0/695322069 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc017070 con 0x7f78ec06c480 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78d37fe640 1 -- 172.21.15.52:0/695322069 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78dc0052d0 con 0x7f78ec06c480 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ec0cc250 con 0x7f78ec06c480 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78d37fe640 1 -- 172.21.15.52:0/695322069 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc003830 con 0x7f78ec06c480 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.842+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ec0cc810 con 0x7f78ec06c480 2024-05-11T01:13:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.843+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ec0cb2f0 con 0x7f78ec06c480 2024-05-11T01:13:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.844+0000 7f78d37fe640 1 -- 172.21.15.52:0/695322069 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f78dc01a070 con 0x7f78ec06c480 2024-05-11T01:13:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.845+0000 7f78d37fe640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f78d80764d0 0x7f78d8078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.845+0000 7f78d37fe640 1 -- 172.21.15.52:0/695322069 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f78dc096d60 con 0x7f78ec06c480 2024-05-11T01:13:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.845+0000 7f78ea7fc640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f78d80764d0 0x7f78d8078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.846+0000 7f78ea7fc640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f78d80764d0 0x7f78d8078990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f78e0034000 tx=0x7f78e002e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:19.849+0000 7f78d37fe640 1 -- 172.21.15.52:0/695322069 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f78ec09bf30 con 0x7f78ec06c480 2024-05-11T01:13:20.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.065+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f78ec0695d0 con 0x7f78d80764d0 2024-05-11T01:13:20.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.138+0000 7f78d37fe640 1 -- 172.21.15.52:0/695322069 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f78ec0695d0 con 0x7f78d80764d0 2024-05-11T01:13:20.138 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f78d80764d0 msgr2=0x7f78d8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f78d80764d0 0x7f78d8078990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f78e0034000 tx=0x7f78e002e000 comp rx=0 tx=0).stop 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec06c480 msgr2=0x7f78ec155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec06c480 0x7f78ec155f70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c3d0 tx=0x7f78dc00c890 comp rx=0 tx=0).stop 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 shutdown_connections 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f78ec15d0e0 0x7f78ec0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ec072330 0x7f78ec0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f78d80764d0 0x7f78d8078990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 --2- 172.21.15.52:0/695322069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78ec06c480 0x7f78ec155f70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 >> 172.21.15.52:0/695322069 conn(0x7f78ec0cd830 msgr2=0x7f78ec0b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 shutdown_connections 2024-05-11T01:13:20.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.140+0000 7f78f1674640 1 -- 172.21.15.52:0/695322069 wait complete. 2024-05-11T01:13:20.150 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2024-05-11T01:13:20.150 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1m 2024-05-11T01:13:20.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.257+0000 7f3ffbea9640 1 Processor -- start 2024-05-11T01:13:20.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.258+0000 7f3ffbea9640 1 -- start start 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.258+0000 7f3ffbea9640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff40d2d60 0x7f3ff40d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.258+0000 7f3ffbea9640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff40d3a90 con 0x7f3ff415f2f0 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.258+0000 7f3ffbea9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff40d3d70 con 0x7f3ff40d4e40 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.258+0000 7f3ffbea9640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40d4050 con 0x7f3ff40d2d60 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.258+0000 7f3ff9c45640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f3ff415f2f0 0x7f3ff40cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:53440/0 (socket says 172.21.15.52:53440) 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.258+0000 7f3ff9c45640 1 -- 172.21.15.52:0/1544273796 learned_addr learned my addr 172.21.15.52:0/1544273796 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ffa446640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff40d2d60 0x7f3ff40d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 543441099 0 0) 0x7f3ff40d3a90 con 0x7f3ff415f2f0 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fdc003520 con 0x7f3ff415f2f0 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1273637256 0 0) 0x7f3ff40d3d70 con 0x7f3ff40d4e40 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ff40d3a90 con 0x7f3ff40d4e40 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 226795014 0 0) 0x7f3fdc003520 con 0x7f3ff415f2f0 2024-05-11T01:13:20.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff40d2d60 msgr2=0x7f3ff40d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff40d2d60 0x7f3ff40d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 >> v1:172.21.15.119:6789/0 conn(0x7f3ff40d4e40 legacy=0x7f3ff40d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff40b4e60 con 0x7f3ff415f2f0 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f3fec0016f0 con 0x7f3ff415f2f0 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 >> v1:172.21.15.52:6789/0 conn(0x7f3ff415f2f0 legacy=0x7f3ff40cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff40d2d60 0x7f3fdc003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fdc004210 0x7f3fdc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fdc004d90 0x7f3fdc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff40d4050 con 0x7f3fdc004210 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00bbd0 con 0x7f3fdc004d90 2024-05-11T01:13:20.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3fdc00bdd0 con 0x7f3ff40d2d60 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff9c45640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff40d2d60 0x7f3fdc003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ff9444640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fdc004210 0x7f3fdc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.259+0000 7f3ffa446640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fdc004d90 0x7f3fdc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.260+0000 7f3ff9c45640 1 -- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fdc004d90 msgr2=0x7f3fdc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.260+0000 7f3ff9c45640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fdc004d90 0x7f3fdc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.260+0000 7f3ff9c45640 1 -- 172.21.15.52:0/1544273796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fdc004210 msgr2=0x7f3fdc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.260+0000 7f3ff9c45640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fdc004210 0x7f3fdc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.260+0000 7f3ff9c45640 1 -- 172.21.15.52:0/1544273796 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc003520 con 0x7f3ff40d2d60 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.260+0000 7f3ff9444640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fdc004210 0x7f3fdc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.260+0000 7f3ff9c45640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff40d2d60 0x7f3fdc003c60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c0e0 tx=0x7f3fdc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e7144da02fa385d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec007c00 con 0x7f3ff40d2d60 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec005af0 con 0x7f3ff40d2d60 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ff8c43640 1 -- 172.21.15.52:0/1544273796 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec007070 con 0x7f3ff40d2d60 2024-05-11T01:13:20.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ffbea9640 1 -- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff40d2d60 msgr2=0x7f3fdc003c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ffbea9640 1 --2- 172.21.15.52:0/1544273796 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff40d2d60 0x7f3fdc003c60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c0e0 tx=0x7f3fdc005830 comp rx=0 tx=0).stop 2024-05-11T01:13:20.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ffa446640 1 -- 172.21.15.52:0/1544273796 reap_dead start 2024-05-11T01:13:20.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ffbea9640 1 -- 172.21.15.52:0/1544273796 shutdown_connections 2024-05-11T01:13:20.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ffbea9640 1 -- 172.21.15.52:0/1544273796 >> 172.21.15.52:0/1544273796 conn(0x7f3ff408ebb0 msgr2=0x7f3ff409e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:20.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ffbea9640 1 -- 172.21.15.52:0/1544273796 shutdown_connections 2024-05-11T01:13:20.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.261+0000 7f3ffbea9640 1 -- 172.21.15.52:0/1544273796 wait complete. 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.262+0000 7f3ffbea9640 1 Processor -- start 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ffbea9640 1 -- start start 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ffbea9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ff40d2d60 0x7f3ff40ba490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ffbea9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff415fa00 0x7f3ff415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ffbea9640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff41ea910 0x7f3ff41ead90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ffbea9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff415f850 con 0x7f3ff415fa00 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ffbea9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff415f6d0 con 0x7f3ff40d2d60 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ffbea9640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40d31e0 con 0x7f3ff41ea910 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ff9444640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff415fa00 0x7f3ff415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ff9444640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff415fa00 0x7f3ff415fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52200/0 (socket says 172.21.15.52:52200) 2024-05-11T01:13:20.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ff9444640 1 -- 172.21.15.52:0/871664491 learned_addr learned my addr 172.21.15.52:0/871664491 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ff9c45640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ff40d2d60 0x7f3ff40ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.263+0000 7f3ffa446640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff41ea910 0x7f3ff41ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ff9444640 1 -- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff41ea910 msgr2=0x7f3ff41ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ff9444640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff41ea910 0x7f3ff41ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ff9444640 1 -- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ff40d2d60 msgr2=0x7f3ff40ba490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ff9444640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ff40d2d60 0x7f3ff40ba490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ff9444640 1 -- 172.21.15.52:0/871664491 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff41eb640 con 0x7f3ff415fa00 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ff9444640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff415fa00 0x7f3ff415fe60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800f960 tx=0x7f3fe800fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ff27fc640 1 -- 172.21.15.52:0/871664491 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80052e0 con 0x7f3ff415fa00 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ff27fc640 1 -- 172.21.15.52:0/871664491 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe8018070 con 0x7f3ff415fa00 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff40c94f0 con 0x7f3ff415fa00 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ff27fc640 1 -- 172.21.15.52:0/871664491 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800da10 con 0x7f3ff415fa00 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.264+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff40c9ab0 con 0x7f3ff415fa00 2024-05-11T01:13:20.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.265+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff40695d0 con 0x7f3ff415fa00 2024-05-11T01:13:20.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.266+0000 7f3ff27fc640 1 -- 172.21.15.52:0/871664491 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f3fe800dbb0 con 0x7f3ff415fa00 2024-05-11T01:13:20.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.267+0000 7f3ff27fc640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3fe0076500 0x7f3fe00789c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.267+0000 7f3ff27fc640 1 -- 172.21.15.52:0/871664491 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f3fe80979f0 con 0x7f3ff415fa00 2024-05-11T01:13:20.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.267+0000 7f3ff9c45640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3fe0076500 0x7f3fe00789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.267+0000 7f3ff9c45640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3fe0076500 0x7f3fe00789c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fec032000 tx=0x7f3fec034040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:20.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.271+0000 7f3ff27fc640 1 -- 172.21.15.52:0/871664491 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3fe80608f0 con 0x7f3ff415fa00 2024-05-11T01:13:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.488+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3ff40d6080 con 0x7f3fe0076500 2024-05-11T01:13:20.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.563+0000 7f3ff27fc640 1 -- 172.21.15.52:0/871664491 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f3ff40d6080 con 0x7f3fe0076500 2024-05-11T01:13:20.562 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2024-05-11T01:13:20.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3fe0076500 msgr2=0x7f3fe00789c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3fe0076500 0x7f3fe00789c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fec032000 tx=0x7f3fec034040 comp rx=0 tx=0).stop 2024-05-11T01:13:20.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff415fa00 msgr2=0x7f3ff415fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff415fa00 0x7f3ff415fe60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800f960 tx=0x7f3fe800fe20 comp rx=0 tx=0).stop 2024-05-11T01:13:20.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 shutdown_connections 2024-05-11T01:13:20.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ff41ea910 0x7f3ff41ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3fe0076500 0x7f3fe00789c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff415fa00 0x7f3ff415fe60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 --2- 172.21.15.52:0/871664491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ff40d2d60 0x7f3ff40ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 >> 172.21.15.52:0/871664491 conn(0x7f3ff408ebb0 msgr2=0x7f3ff40d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:20.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 shutdown_connections 2024-05-11T01:13:20.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.565+0000 7f3ffbea9640 1 -- 172.21.15.52:0/871664491 wait complete. 2024-05-11T01:13:20.575 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2024-05-11T01:13:20.575 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1m 2024-05-11T01:13:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.682+0000 7fd796093640 1 Processor -- start 2024-05-11T01:13:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.682+0000 7fd796093640 1 -- start start 2024-05-11T01:13:20.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.683+0000 7fd796093640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7900d1970 0x7fd7900d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.683+0000 7fd796093640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7900cfc60 con 0x7fd7900d0700 2024-05-11T01:13:20.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.683+0000 7fd796093640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd790163120 con 0x7fd79015f430 2024-05-11T01:13:20.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.683+0000 7fd796093640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd7900d2a80 con 0x7fd7900d1970 2024-05-11T01:13:20.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.683+0000 7fd78ffff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd7900d0700 0x7fd7900d4e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:53450/0 (socket says 172.21.15.52:53450) 2024-05-11T01:13:20.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.683+0000 7fd78ffff640 1 -- 172.21.15.52:0/594560023 learned_addr learned my addr 172.21.15.52:0/594560023 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.683+0000 7fd78effd640 1 --2- 172.21.15.52:0/594560023 >> v2:172.21.15.119:3300/0 conn(0x7fd7900d1970 0x7fd7900d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.683+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3256282020 0 0) 0x7fd7900cfc60 con 0x7fd7900d0700 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd778003520 con 0x7fd7900d0700 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 437419197 0 0) 0x7fd778003520 con 0x7fd7900d0700 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 >> v2:172.21.15.119:3300/0 conn(0x7fd7900d1970 msgr2=0x7fd7900d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 --2- 172.21.15.52:0/594560023 >> v2:172.21.15.119:3300/0 conn(0x7fd7900d1970 0x7fd7900d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 >> v1:172.21.15.119:6789/0 conn(0x7fd79015f430 legacy=0x7fd7900b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7900d24c0 con 0x7fd7900d0700 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fd788001a70 con 0x7fd7900d0700 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 >> v1:172.21.15.52:6789/0 conn(0x7fd7900d0700 legacy=0x7fd7900d4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd778003a00 0x7fd778003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778004800 0x7fd778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7780051c0 0x7fd77800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900d2a80 con 0x7fd778003a00 2024-05-11T01:13:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd77800c060 con 0x7fd778004800 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd77800c260 con 0x7fd7780051c0 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78f7fe640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd778003a00 0x7fd778003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78effd640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778004800 0x7fd778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78ffff640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7780051c0 0x7fd77800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78f7fe640 1 -- 172.21.15.52:0/594560023 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7780051c0 msgr2=0x7fd77800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78f7fe640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7780051c0 0x7fd77800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78f7fe640 1 -- 172.21.15.52:0/594560023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778004800 msgr2=0x7fd778004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78f7fe640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778004800 0x7fd778004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78f7fe640 1 -- 172.21.15.52:0/594560023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd778003520 con 0x7fd778003a00 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.684+0000 7fd78ffff640 1 -- 172.21.15.52:0/594560023 reap_dead start 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.685+0000 7fd78f7fe640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd778003a00 0x7fd778003de0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd780001890 tx=0x7fd778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f6a8cb49adf5f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.685+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780037070 con 0x7fd778003a00 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.685+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78002ac20 con 0x7fd778003a00 2024-05-11T01:13:20.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.685+0000 7fd78e7fc640 1 -- 172.21.15.52:0/594560023 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7800014b0 con 0x7fd778003a00 2024-05-11T01:13:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.686+0000 7fd796093640 1 -- 172.21.15.52:0/594560023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd778003a00 msgr2=0x7fd778003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.686+0000 7fd796093640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd778003a00 0x7fd778003de0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd780001890 tx=0x7fd778005830 comp rx=0 tx=0).stop 2024-05-11T01:13:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.686+0000 7fd796093640 1 -- 172.21.15.52:0/594560023 shutdown_connections 2024-05-11T01:13:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.686+0000 7fd796093640 1 --2- 172.21.15.52:0/594560023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd778003a00 0x7fd778003de0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.686+0000 7fd796093640 1 -- 172.21.15.52:0/594560023 >> 172.21.15.52:0/594560023 conn(0x7fd79008ebb0 msgr2=0x7fd79009e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.686+0000 7fd796093640 1 -- 172.21.15.52:0/594560023 shutdown_connections 2024-05-11T01:13:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.686+0000 7fd796093640 1 -- 172.21.15.52:0/594560023 wait complete. 2024-05-11T01:13:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.687+0000 7fd796093640 1 Processor -- start 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.687+0000 7fd796093640 1 -- start start 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd796093640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778003a00 0x7fd7900ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd796093640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd79006e3c0 0x7fd79006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd796093640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd790151f30 0x7fd7901523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd796093640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900d01e0 con 0x7fd790151f30 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd796093640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900cff00 con 0x7fd778003a00 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd796093640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd7900d1de0 con 0x7fd79006e3c0 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd790151f30 0x7fd7901523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd790151f30 0x7fd7901523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52204/0 (socket says 172.21.15.52:52204) 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78ffff640 1 -- 172.21.15.52:0/3074214807 learned_addr learned my addr 172.21.15.52:0/3074214807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78f7fe640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778003a00 0x7fd7900ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78effd640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd79006e3c0 0x7fd79006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78ffff640 1 -- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd79006e3c0 msgr2=0x7fd79006e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78ffff640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd79006e3c0 0x7fd79006e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78ffff640 1 -- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778003a00 msgr2=0x7fd7900ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78ffff640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778003a00 0x7fd7900ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78ffff640 1 -- 172.21.15.52:0/3074214807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7900c9ab0 con 0x7fd790151f30 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78f7fe640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778003a00 0x7fd7900ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.688+0000 7fd78ffff640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd790151f30 0x7fd7901523b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd79008efe0 tx=0x7fd788007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.689+0000 7fd777fff640 1 -- 172.21.15.52:0/3074214807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78800ea40 con 0x7fd790151f30 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.689+0000 7fd777fff640 1 -- 172.21.15.52:0/3074214807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd788018070 con 0x7fd790151f30 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.689+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7900c9d90 con 0x7fd790151f30 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.689+0000 7fd777fff640 1 -- 172.21.15.52:0/3074214807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78800ee40 con 0x7fd790151f30 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.689+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7901528f0 con 0x7fd790151f30 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.690+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7900695d0 con 0x7fd790151f30 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.690+0000 7fd777fff640 1 -- 172.21.15.52:0/3074214807 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fd78801d460 con 0x7fd790151f30 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.691+0000 7fd777fff640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd77c0764d0 0x7fd77c078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.691+0000 7fd777fff640 1 -- 172.21.15.52:0/3074214807 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7fd788097c40 con 0x7fd790151f30 2024-05-11T01:13:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.691+0000 7fd78f7fe640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd77c0764d0 0x7fd77c078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.692+0000 7fd78f7fe640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd77c0764d0 0x7fd77c078990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd780034000 tx=0x7fd780009030 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.695+0000 7fd777fff640 1 -- 172.21.15.52:0/3074214807 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd7880612b0 con 0x7fd790151f30 2024-05-11T01:13:20.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.912+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd790152cd0 con 0x7fd77c0764d0 2024-05-11T01:13:20.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.985+0000 7fd777fff640 1 -- 172.21.15.52:0/3074214807 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd78002ad40 con 0x7fd77c0764d0 2024-05-11T01:13:20.984 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd77c0764d0 msgr2=0x7fd77c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd77c0764d0 0x7fd77c078990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd780034000 tx=0x7fd780009030 comp rx=0 tx=0).stop 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd790151f30 msgr2=0x7fd7901523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd790151f30 0x7fd7901523b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd79008efe0 tx=0x7fd788007bc0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 shutdown_connections 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd77c0764d0 0x7fd77c078990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd790151f30 0x7fd7901523b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd79006e3c0 0x7fd79006e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 --2- 172.21.15.52:0/3074214807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd778003a00 0x7fd7900ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 >> 172.21.15.52:0/3074214807 conn(0x7fd79008ebb0 msgr2=0x7fd79006d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 shutdown_connections 2024-05-11T01:13:20.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:20.987+0000 7fd796093640 1 -- 172.21.15.52:0/3074214807 wait complete. 2024-05-11T01:13:20.997 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2024-05-11T01:13:20.997 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1m 2024-05-11T01:13:21.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.106+0000 7f6f53790640 1 Processor -- start 2024-05-11T01:13:21.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.106+0000 7f6f53790640 1 -- start start 2024-05-11T01:13:21.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.107+0000 7f6f53790640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6f4c162990 0x7f6f4c0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.107+0000 7f6f53790640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c0d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.107+0000 7f6f53790640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f4c0d3390 con 0x7f6f4c0d3b90 2024-05-11T01:13:21.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.107+0000 7f6f53790640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4c0d3fe0 con 0x7f6f4c162990 2024-05-11T01:13:21.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.107+0000 7f6f53790640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c0b7f50 con 0x7f6f4c0d0740 2024-05-11T01:13:21.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.107+0000 7f6f50d2b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c0d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.107+0000 7f6f5152c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6f4c162990 0x7f6f4c0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f51d2d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6f4c0d3b90 0x7f6f4c163000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:46974/0 (socket says 172.21.15.52:46974) 2024-05-11T01:13:21.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f5152c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6f4c162990 0x7f6f4c0d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:47282/0 (socket says 172.21.15.52:47282) 2024-05-11T01:13:21.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f51d2d640 1 -- 172.21.15.52:0/10857499 learned_addr learned my addr 172.21.15.52:0/10857499 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f43fff640 1 -- 172.21.15.52:0/10857499 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 823436918 0 0) 0x7f6f4c0d3390 con 0x7f6f4c0d3b90 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f43fff640 1 -- 172.21.15.52:0/10857499 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f38003520 con 0x7f6f4c0d3b90 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f5152c640 1 -- 172.21.15.52:0/10857499 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 msgr2=0x7f6f4c0d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f5152c640 1 --2- 172.21.15.52:0/10857499 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c0d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f5152c640 1 -- 172.21.15.52:0/10857499 >> v1:172.21.15.119:6789/0 conn(0x7f6f4c0d3b90 legacy=0x7f6f4c163000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f5152c640 1 -- 172.21.15.52:0/10857499 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c162d70 con 0x7f6f4c162990 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.108+0000 7f6f50d2b640 1 --2- 172.21.15.52:0/10857499 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c0d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f5152c640 1 --2- 172.21.15.52:0/10857499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4c162990 0x7f6f4c0d2e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f480097a0 tx=0x7f6f4802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0a9ac71c4534a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f43fff640 1 -- 172.21.15.52:0/10857499 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48037070 con 0x7f6f4c162990 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f43fff640 1 -- 172.21.15.52:0/10857499 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f4802ae50 con 0x7f6f4c162990 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f43fff640 1 -- 172.21.15.52:0/10857499 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48032640 con 0x7f6f4c162990 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f53790640 1 -- 172.21.15.52:0/10857499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4c162990 msgr2=0x7f6f4c0d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f53790640 1 --2- 172.21.15.52:0/10857499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4c162990 0x7f6f4c0d2e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f480097a0 tx=0x7f6f4802a100 comp rx=0 tx=0).stop 2024-05-11T01:13:21.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f53790640 1 -- 172.21.15.52:0/10857499 shutdown_connections 2024-05-11T01:13:21.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f53790640 1 --2- 172.21.15.52:0/10857499 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c0d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f53790640 1 --2- 172.21.15.52:0/10857499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4c162990 0x7f6f4c0d2e50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.109+0000 7f6f53790640 1 -- 172.21.15.52:0/10857499 >> 172.21.15.52:0/10857499 conn(0x7f6f4c09e490 msgr2=0x7f6f4c0d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:21.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.110+0000 7f6f53790640 1 -- 172.21.15.52:0/10857499 shutdown_connections 2024-05-11T01:13:21.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.110+0000 7f6f53790640 1 -- 172.21.15.52:0/10857499 wait complete. 2024-05-11T01:13:21.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.111+0000 7f6f53790640 1 Processor -- start 2024-05-11T01:13:21.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.111+0000 7f6f53790640 1 -- start start 2024-05-11T01:13:21.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.111+0000 7f6f53790640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.111+0000 7f6f53790640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4c0d3b90 0x7f6f4c0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.111+0000 7f6f53790640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4c162990 0x7f6f4c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f53790640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c0d0e80 con 0x7f6f4c162990 2024-05-11T01:13:21.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f53790640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c0d5d40 con 0x7f6f4c0d3b90 2024-05-11T01:13:21.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f53790640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c0b81a0 con 0x7f6f4c0d0740 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f51d2d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4c162990 0x7f6f4c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f5152c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f5152c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c15d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:44710/0 (socket says 172.21.15.52:44710) 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f51d2d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4c162990 0x7f6f4c15dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52210/0 (socket says 172.21.15.52:52210) 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f5152c640 1 -- 172.21.15.52:0/3261848589 learned_addr learned my addr 172.21.15.52:0/3261848589 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f50d2b640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4c0d3b90 0x7f6f4c0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f51d2d640 1 -- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 msgr2=0x7f6f4c15d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f51d2d640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c15d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f51d2d640 1 -- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4c0d3b90 msgr2=0x7f6f4c0cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f51d2d640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4c0d3b90 0x7f6f4c0cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f51d2d640 1 -- 172.21.15.52:0/3261848589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c0cc140 con 0x7f6f4c162990 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f51d2d640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4c162990 0x7f6f4c15dc80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400cab0 tx=0x7f6f4400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f41ffb640 1 -- 172.21.15.52:0/3261848589 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4400ac30 con 0x7f6f4c162990 2024-05-11T01:13:21.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f41ffb640 1 -- 172.21.15.52:0/3261848589 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f4400add0 con 0x7f6f4c162990 2024-05-11T01:13:21.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c0cc420 con 0x7f6f4c162990 2024-05-11T01:13:21.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.112+0000 7f6f41ffb640 1 -- 172.21.15.52:0/3261848589 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44012600 con 0x7f6f4c162990 2024-05-11T01:13:21.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.113+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c0cc9e0 con 0x7f6f4c162990 2024-05-11T01:13:21.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.114+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c0695d0 con 0x7f6f4c162990 2024-05-11T01:13:21.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.114+0000 7f6f41ffb640 1 -- 172.21.15.52:0/3261848589 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f6f440127a0 con 0x7f6f4c162990 2024-05-11T01:13:21.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.114+0000 7f6f41ffb640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6f300764d0 0x7f6f30078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.115+0000 7f6f41ffb640 1 -- 172.21.15.52:0/3261848589 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f6f44097ea0 con 0x7f6f4c162990 2024-05-11T01:13:21.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.115+0000 7f6f5152c640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6f300764d0 0x7f6f30078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.115+0000 7f6f5152c640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6f300764d0 0x7f6f30078990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6f48000c00 tx=0x7f6f4802e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:21.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.119+0000 7f6f41ffb640 1 -- 172.21.15.52:0/3261848589 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6f44060ec0 con 0x7f6f4c162990 2024-05-11T01:13:21.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.336+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f6f4c161f20 con 0x7f6f300764d0 2024-05-11T01:13:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.406+0000 7f6f41ffb640 1 -- 172.21.15.52:0/3261848589 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f6f4c161f20 con 0x7f6f300764d0 2024-05-11T01:13:21.406 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2024-05-11T01:13:21.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6f300764d0 msgr2=0x7f6f30078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6f300764d0 0x7f6f30078990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6f48000c00 tx=0x7f6f4802e000 comp rx=0 tx=0).stop 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4c162990 msgr2=0x7f6f4c15dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4c162990 0x7f6f4c15dc80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400cab0 tx=0x7f6f4400cf70 comp rx=0 tx=0).stop 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 shutdown_connections 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f6f300764d0 0x7f6f30078990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4c162990 0x7f6f4c15dc80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4c0d3b90 0x7f6f4c0cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 --2- 172.21.15.52:0/3261848589 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6f4c0d0740 0x7f6f4c15d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 >> 172.21.15.52:0/3261848589 conn(0x7f6f4c09e490 msgr2=0x7f6f4c0ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 shutdown_connections 2024-05-11T01:13:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.409+0000 7f6f53790640 1 -- 172.21.15.52:0/3261848589 wait complete. 2024-05-11T01:13:21.419 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2024-05-11T01:13:21.419 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1m 2024-05-11T01:13:21.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.526+0000 7f2e18475640 1 Processor -- start 2024-05-11T01:13:21.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.526+0000 7f2e18475640 1 -- start start 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.527+0000 7f2e18475640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e1015f2f0 0x7f2e100cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.527+0000 7f2e18475640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e100d2d60 0x7f2e100d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.527+0000 7f2e18475640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e100d3a90 con 0x7f2e100d4e40 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.527+0000 7f2e18475640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f2e100d3d70 con 0x7f2e1015f2f0 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.527+0000 7f2e18475640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2e100d4050 con 0x7f2e100d2d60 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.527+0000 7f2e16211640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f2e1015f2f0 0x7f2e100cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.528+0000 7f2e15a10640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2e100d4e40 0x7f2e100d5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:53458/0 (socket says 172.21.15.52:53458) 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.528+0000 7f2e15a10640 1 -- 172.21.15.52:0/3081840542 learned_addr learned my addr 172.21.15.52:0/3081840542 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.528+0000 7f2e16a12640 1 --2- 172.21.15.52:0/3081840542 >> v2:172.21.15.119:3300/0 conn(0x7f2e100d2d60 0x7f2e100d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.528+0000 7f2e16211640 1 -- 172.21.15.52:0/3081840542 >> v2:172.21.15.119:3300/0 conn(0x7f2e100d2d60 msgr2=0x7f2e100d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.528+0000 7f2e16211640 1 --2- 172.21.15.52:0/3081840542 >> v2:172.21.15.119:3300/0 conn(0x7f2e100d2d60 0x7f2e100d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.528+0000 7f2e16211640 1 -- 172.21.15.52:0/3081840542 >> v1:172.21.15.52:6789/0 conn(0x7f2e100d4e40 legacy=0x7f2e100d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.528+0000 7f2e16211640 1 -- 172.21.15.52:0/3081840542 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e100b4e60 con 0x7f2e1015f2f0 2024-05-11T01:13:21.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.528+0000 7f2e1520f640 1 -- 172.21.15.52:0/3081840542 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 560959993 0 0) 0x7f2e100d3a90 con 0x7f2e100d4e40 2024-05-11T01:13:21.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e16211640 1 --2- 172.21.15.52:0/3081840542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e1015f2f0 0x7f2e100cfca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2e100c8c60 tx=0x7f2e0002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b5411f92b9a020 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:21.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e1520f640 1 -- 172.21.15.52:0/3081840542 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00007740 con 0x7f2e1015f2f0 2024-05-11T01:13:21.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e1520f640 1 -- 172.21.15.52:0/3081840542 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e000078e0 con 0x7f2e1015f2f0 2024-05-11T01:13:21.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e1520f640 1 -- 172.21.15.52:0/3081840542 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00007cb0 con 0x7f2e1015f2f0 2024-05-11T01:13:21.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e18475640 1 -- 172.21.15.52:0/3081840542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e1015f2f0 msgr2=0x7f2e100cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e18475640 1 --2- 172.21.15.52:0/3081840542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e1015f2f0 0x7f2e100cfca0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2e100c8c60 tx=0x7f2e0002a680 comp rx=0 tx=0).stop 2024-05-11T01:13:21.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e18475640 1 -- 172.21.15.52:0/3081840542 shutdown_connections 2024-05-11T01:13:21.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e18475640 1 --2- 172.21.15.52:0/3081840542 >> v2:172.21.15.119:3300/0 conn(0x7f2e100d2d60 0x7f2e100d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e18475640 1 --2- 172.21.15.52:0/3081840542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e1015f2f0 0x7f2e100cfca0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e18475640 1 -- 172.21.15.52:0/3081840542 >> 172.21.15.52:0/3081840542 conn(0x7f2e1008ebb0 msgr2=0x7f2e1009e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:21.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e18475640 1 -- 172.21.15.52:0/3081840542 shutdown_connections 2024-05-11T01:13:21.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.529+0000 7f2e18475640 1 -- 172.21.15.52:0/3081840542 wait complete. 2024-05-11T01:13:21.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.531+0000 7f2e18475640 1 Processor -- start 2024-05-11T01:13:21.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.531+0000 7f2e18475640 1 -- start start 2024-05-11T01:13:21.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.531+0000 7f2e18475640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e100d2d60 0x7f2e1015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.531+0000 7f2e18475640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e100d4e40 0x7f2e1015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.531+0000 7f2e18475640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1015f2f0 0x7f2e1015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.531+0000 7f2e18475640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e100d01e0 con 0x7f2e100d4e40 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.531+0000 7f2e18475640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e100d3720 con 0x7f2e1015f2f0 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.531+0000 7f2e18475640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f2e100d0360 con 0x7f2e100d2d60 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e15a10640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e100d4e40 0x7f2e1015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e15a10640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e100d4e40 0x7f2e1015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52228/0 (socket says 172.21.15.52:52228) 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e15a10640 1 -- 172.21.15.52:0/2390474271 learned_addr learned my addr 172.21.15.52:0/2390474271 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e16a12640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1015f2f0 0x7f2e1015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e16211640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e100d2d60 0x7f2e1015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e15a10640 1 -- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e100d2d60 msgr2=0x7f2e1015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e15a10640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e100d2d60 0x7f2e1015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e15a10640 1 -- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1015f2f0 msgr2=0x7f2e1015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e15a10640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1015f2f0 0x7f2e1015d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e15a10640 1 -- 172.21.15.52:0/2390474271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e100cd0d0 con 0x7f2e100d4e40 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e15a10640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e100d4e40 0x7f2e1015d380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400c490 tx=0x7f2e0400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e0effd640 1 -- 172.21.15.52:0/2390474271 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04018070 con 0x7f2e100d4e40 2024-05-11T01:13:21.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e0effd640 1 -- 172.21.15.52:0/2390474271 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e04007500 con 0x7f2e100d4e40 2024-05-11T01:13:21.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e0effd640 1 -- 172.21.15.52:0/2390474271 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04004a20 con 0x7f2e100d4e40 2024-05-11T01:13:21.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e100cd3b0 con 0x7f2e100d4e40 2024-05-11T01:13:21.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.532+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e100cd970 con 0x7f2e100d4e40 2024-05-11T01:13:21.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.533+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e100695d0 con 0x7f2e100d4e40 2024-05-11T01:13:21.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.534+0000 7f2e0effd640 1 -- 172.21.15.52:0/2390474271 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f2e0401b070 con 0x7f2e100d4e40 2024-05-11T01:13:21.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.534+0000 7f2e0effd640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2dfc076500 0x7f2dfc0789c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.534+0000 7f2e0effd640 1 -- 172.21.15.52:0/2390474271 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f2e04096d90 con 0x7f2e100d4e40 2024-05-11T01:13:21.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.535+0000 7f2e16211640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2dfc076500 0x7f2dfc0789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.535+0000 7f2e16211640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2dfc076500 0x7f2dfc0789c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2e00034000 tx=0x7f2e0002f000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:21.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.539+0000 7f2e0effd640 1 -- 172.21.15.52:0/2390474271 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2e04060360 con 0x7f2e100d4e40 2024-05-11T01:13:21.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.754+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f2e100d6080 con 0x7f2dfc076500 2024-05-11T01:13:21.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.826+0000 7f2e0effd640 1 -- 172.21.15.52:0/2390474271 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f2e100d6080 con 0x7f2dfc076500 2024-05-11T01:13:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2024-05-11T01:13:21.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2dfc076500 msgr2=0x7f2dfc0789c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2dfc076500 0x7f2dfc0789c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2e00034000 tx=0x7f2e0002f000 comp rx=0 tx=0).stop 2024-05-11T01:13:21.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e100d4e40 msgr2=0x7f2e1015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e100d4e40 0x7f2e1015d380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400c490 tx=0x7f2e0400c950 comp rx=0 tx=0).stop 2024-05-11T01:13:21.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 shutdown_connections 2024-05-11T01:13:21.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1015f2f0 0x7f2e1015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2dfc076500 0x7f2dfc0789c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e100d4e40 0x7f2e1015d380 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 --2- 172.21.15.52:0/2390474271 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2e100d2d60 0x7f2e1015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.828+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 >> 172.21.15.52:0/2390474271 conn(0x7f2e1008ebb0 msgr2=0x7f2e100b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:21.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.829+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 shutdown_connections 2024-05-11T01:13:21.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.829+0000 7f2e18475640 1 -- 172.21.15.52:0/2390474271 wait complete. 2024-05-11T01:13:21.838 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2024-05-11T01:13:21.839 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1m 2024-05-11T01:13:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.946+0000 7f5a6f872640 1 Processor -- start 2024-05-11T01:13:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.946+0000 7f5a6f872640 1 -- start start 2024-05-11T01:13:21.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.947+0000 7f5a6f872640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a6815f430 0x7f5a680cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.947+0000 7f5a6f872640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5a680d1a30 0x7f5a680cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.947+0000 7f5a6f872640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5a68071690 0x7f5a68073270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.947+0000 7f5a6f872640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6815f810 con 0x7f5a680d1a30 2024-05-11T01:13:21.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.947+0000 7f5a6f872640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5a681632b0 con 0x7f5a68071690 2024-05-11T01:13:21.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.947+0000 7f5a6f872640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5a68163590 con 0x7f5a6815f430 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.947+0000 7f5a6ce0d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5a680d1a30 0x7f5a680cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.947+0000 7f5a6ce0d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5a680d1a30 0x7f5a680cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49938/0 (socket says 172.21.15.52:49938) 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.947+0000 7f5a6ce0d640 1 -- 172.21.15.52:0/586724400 learned_addr learned my addr 172.21.15.52:0/586724400 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.948+0000 7f5a6de0f640 1 --2- 172.21.15.52:0/586724400 >> v2:172.21.15.119:3300/0 conn(0x7f5a68071690 0x7f5a68073270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.948+0000 7f5a6ce0d640 1 -- 172.21.15.52:0/586724400 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a6815f430 msgr2=0x7f5a680cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.948+0000 7f5a6d60e640 1 --2- 172.21.15.52:0/586724400 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a6815f430 0x7f5a680cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.948+0000 7f5a6ce0d640 1 --2- 172.21.15.52:0/586724400 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a6815f430 0x7f5a680cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.948+0000 7f5a6ce0d640 1 -- 172.21.15.52:0/586724400 >> v2:172.21.15.119:3300/0 conn(0x7f5a68071690 msgr2=0x7f5a68073270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.948+0000 7f5a6ce0d640 1 --2- 172.21.15.52:0/586724400 >> v2:172.21.15.119:3300/0 conn(0x7f5a68071690 0x7f5a68073270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.948+0000 7f5a6ce0d640 1 -- 172.21.15.52:0/586724400 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a68163870 con 0x7f5a680d1a30 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.948+0000 7f5a6de0f640 1 --2- 172.21.15.52:0/586724400 >> v2:172.21.15.119:3300/0 conn(0x7f5a68071690 0x7f5a68073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.948+0000 7f5a6ce0d640 1 --2- 172.21.15.52:0/586724400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a680d1a30 0x7f5a680cca50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a68071e00 tx=0x7f5a5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd7f75c1a05abbb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a5ffff640 1 -- 172.21.15.52:0/586724400 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58037070 con 0x7f5a680d1a30 2024-05-11T01:13:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a5ffff640 1 -- 172.21.15.52:0/586724400 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a58032440 con 0x7f5a680d1a30 2024-05-11T01:13:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a5ffff640 1 -- 172.21.15.52:0/586724400 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58032810 con 0x7f5a680d1a30 2024-05-11T01:13:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a6f872640 1 -- 172.21.15.52:0/586724400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a680d1a30 msgr2=0x7f5a680cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a6f872640 1 --2- 172.21.15.52:0/586724400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a680d1a30 0x7f5a680cca50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a68071e00 tx=0x7f5a5802a140 comp rx=0 tx=0).stop 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a6f872640 1 -- 172.21.15.52:0/586724400 shutdown_connections 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a6f872640 1 --2- 172.21.15.52:0/586724400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a680d1a30 0x7f5a680cca50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a6f872640 1 --2- 172.21.15.52:0/586724400 >> v2:172.21.15.119:3300/0 conn(0x7f5a68071690 0x7f5a68073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a6f872640 1 --2- 172.21.15.52:0/586724400 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a6815f430 0x7f5a680cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a6f872640 1 -- 172.21.15.52:0/586724400 >> 172.21.15.52:0/586724400 conn(0x7f5a680ce940 msgr2=0x7f5a680ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a6f872640 1 -- 172.21.15.52:0/586724400 shutdown_connections 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.949+0000 7f5a6f872640 1 -- 172.21.15.52:0/586724400 wait complete. 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.950+0000 7f5a6f872640 1 Processor -- start 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.950+0000 7f5a6f872640 1 -- start start 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6f872640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a68071690 0x7f5a6814d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6f872640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a680d1a30 0x7f5a680c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6f872640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a6815f430 0x7f5a680c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6f872640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68163030 con 0x7f5a6815f430 2024-05-11T01:13:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6f872640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68162eb0 con 0x7f5a680d1a30 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6f872640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5a680737b0 con 0x7f5a68071690 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6de0f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a6815f430 0x7f5a680c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6de0f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a6815f430 0x7f5a680c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49954/0 (socket says 172.21.15.52:49954) 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6de0f640 1 -- 172.21.15.52:0/1697536245 learned_addr learned my addr 172.21.15.52:0/1697536245 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6d60e640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a68071690 0x7f5a6814d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.951+0000 7f5a6ce0d640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a680d1a30 0x7f5a680c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a6de0f640 1 -- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a68071690 msgr2=0x7f5a6814d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a6de0f640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a68071690 0x7f5a6814d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a6de0f640 1 -- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a680d1a30 msgr2=0x7f5a680c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a6de0f640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a680d1a30 0x7f5a680c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a6de0f640 1 -- 172.21.15.52:0/1697536245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6815e430 con 0x7f5a6815f430 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a6de0f640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a6815f430 0x7f5a680c6a80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000cdf0 tx=0x7f5a6000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a5dffb640 1 -- 172.21.15.52:0/1697536245 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60003b90 con 0x7f5a6815f430 2024-05-11T01:13:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a5dffb640 1 -- 172.21.15.52:0/1697536245 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a60003d50 con 0x7f5a6815f430 2024-05-11T01:13:21.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a680c70b0 con 0x7f5a6815f430 2024-05-11T01:13:21.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a5dffb640 1 -- 172.21.15.52:0/1697536245 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60004260 con 0x7f5a6815f430 2024-05-11T01:13:21.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.952+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a680c7670 con 0x7f5a6815f430 2024-05-11T01:13:21.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.953+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a680695d0 con 0x7f5a6815f430 2024-05-11T01:13:21.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.954+0000 7f5a5dffb640 1 -- 172.21.15.52:0/1697536245 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f5a60004400 con 0x7f5a6815f430 2024-05-11T01:13:21.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.954+0000 7f5a5dffb640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5a4c0764d0 0x7f5a4c078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:21.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.954+0000 7f5a5dffb640 1 -- 172.21.15.52:0/1697536245 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f5a60098370 con 0x7f5a6815f430 2024-05-11T01:13:21.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.955+0000 7f5a6d60e640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5a4c0764d0 0x7f5a4c078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:21.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.955+0000 7f5a6d60e640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5a4c0764d0 0x7f5a4c078990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a64006fd0 tx=0x7f5a64008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:21.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:21.959+0000 7f5a5dffb640 1 -- 172.21.15.52:0/1697536245 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5a60061940 con 0x7f5a6815f430 2024-05-11T01:13:22.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.178+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f5a680d0d80 con 0x7f5a4c0764d0 2024-05-11T01:13:22.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.248+0000 7f5a5dffb640 1 -- 172.21.15.52:0/1697536245 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5a680d0d80 con 0x7f5a4c0764d0 2024-05-11T01:13:22.247 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2024-05-11T01:13:22.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5a4c0764d0 msgr2=0x7f5a4c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5a4c0764d0 0x7f5a4c078990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a64006fd0 tx=0x7f5a64008040 comp rx=0 tx=0).stop 2024-05-11T01:13:22.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a6815f430 msgr2=0x7f5a680c6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a6815f430 0x7f5a680c6a80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000cdf0 tx=0x7f5a6000a710 comp rx=0 tx=0).stop 2024-05-11T01:13:22.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 shutdown_connections 2024-05-11T01:13:22.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5a4c0764d0 0x7f5a4c078990 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a6815f430 0x7f5a680c6a80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a680d1a30 0x7f5a680c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 --2- 172.21.15.52:0/1697536245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5a68071690 0x7f5a6814d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 >> 172.21.15.52:0/1697536245 conn(0x7f5a680ce940 msgr2=0x7f5a680c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:22.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 shutdown_connections 2024-05-11T01:13:22.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.251+0000 7f5a6f872640 1 -- 172.21.15.52:0/1697536245 wait complete. 2024-05-11T01:13:22.261 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2024-05-11T01:13:22.261 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1m 2024-05-11T01:13:22.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.369+0000 7f3911b9d640 1 Processor -- start 2024-05-11T01:13:22.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.369+0000 7f3911b9d640 1 -- start start 2024-05-11T01:13:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.370+0000 7f3911b9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f390c15f430 0x7f390c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.370+0000 7f3911b9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f390c0d0700 0x7f390c0d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.370+0000 7f3911b9d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f390c0cfc60 con 0x7f390c0b5420 2024-05-11T01:13:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.370+0000 7f3911b9d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f390c0d2a80 con 0x7f390c0d0700 2024-05-11T01:13:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.370+0000 7f3911b9d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f390c163120 con 0x7f390c15f430 2024-05-11T01:13:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.370+0000 7f390bfff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f390c0d0700 0x7f390c0d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.370+0000 7f390bfff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f390c0d0700 0x7f390c0d0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49956/0 (socket says 172.21.15.52:49956) 2024-05-11T01:13:22.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.370+0000 7f390bfff640 1 -- 172.21.15.52:0/2122822342 learned_addr learned my addr 172.21.15.52:0/2122822342 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:22.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.371+0000 7f390b7fe640 1 --2- 172.21.15.52:0/2122822342 >> v2:172.21.15.119:3300/0 conn(0x7f390c15f430 0x7f390c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.371+0000 7f390bfff640 1 -- 172.21.15.52:0/2122822342 >> v2:172.21.15.119:3300/0 conn(0x7f390c15f430 msgr2=0x7f390c0b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:22.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.371+0000 7f390bfff640 1 --2- 172.21.15.52:0/2122822342 >> v2:172.21.15.119:3300/0 conn(0x7f390c15f430 0x7f390c0b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.371+0000 7f390bfff640 1 -- 172.21.15.52:0/2122822342 >> v1:172.21.15.119:6789/0 conn(0x7f390c0b5420 legacy=0x7f390c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:22.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.371+0000 7f390bfff640 1 -- 172.21.15.52:0/2122822342 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c0d24c0 con 0x7f390c0d0700 2024-05-11T01:13:22.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.371+0000 7f390bfff640 1 --2- 172.21.15.52:0/2122822342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c0d0700 0x7f390c0d0b70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f390c0d6290 tx=0x7f390402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78951b752828d466 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:22.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f390a7fc640 1 -- 172.21.15.52:0/2122822342 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904036070 con 0x7f390c0d0700 2024-05-11T01:13:22.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f390a7fc640 1 -- 172.21.15.52:0/2122822342 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3904031600 con 0x7f390c0d0700 2024-05-11T01:13:22.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f390a7fc640 1 -- 172.21.15.52:0/2122822342 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39040319d0 con 0x7f390c0d0700 2024-05-11T01:13:22.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f3911b9d640 1 -- 172.21.15.52:0/2122822342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c0d0700 msgr2=0x7f390c0d0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f3911b9d640 1 --2- 172.21.15.52:0/2122822342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c0d0700 0x7f390c0d0b70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f390c0d6290 tx=0x7f390402a730 comp rx=0 tx=0).stop 2024-05-11T01:13:22.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f3911b9d640 1 -- 172.21.15.52:0/2122822342 shutdown_connections 2024-05-11T01:13:22.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f3911b9d640 1 --2- 172.21.15.52:0/2122822342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c0d0700 0x7f390c0d0b70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f3911b9d640 1 --2- 172.21.15.52:0/2122822342 >> v2:172.21.15.119:3300/0 conn(0x7f390c15f430 0x7f390c0b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f3911b9d640 1 -- 172.21.15.52:0/2122822342 >> 172.21.15.52:0/2122822342 conn(0x7f390c08ebb0 msgr2=0x7f390c09e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:22.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f3911b9d640 1 -- 172.21.15.52:0/2122822342 shutdown_connections 2024-05-11T01:13:22.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.372+0000 7f3911b9d640 1 -- 172.21.15.52:0/2122822342 wait complete. 2024-05-11T01:13:22.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.373+0000 7f3911b9d640 1 Processor -- start 2024-05-11T01:13:22.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.374+0000 7f3911b9d640 1 -- start start 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.374+0000 7f3911b9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f390c0b5420 0x7f390c0b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.374+0000 7f3911b9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f390c0d0700 0x7f390c162120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.374+0000 7f3911b9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c15f430 0x7f390c147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.374+0000 7f3911b9d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f390c0d01e0 con 0x7f390c15f430 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.374+0000 7f3911b9d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f390c0cff00 con 0x7f390c0d0700 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.374+0000 7f3911b9d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f390c0d2080 con 0x7f390c0b5420 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c15f430 0x7f390c147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c15f430 0x7f390c147ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49968/0 (socket says 172.21.15.52:49968) 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390bfff640 1 -- 172.21.15.52:0/287058001 learned_addr learned my addr 172.21.15.52:0/287058001 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390b7fe640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f390c0b5420 0x7f390c0b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390affd640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f390c0d0700 0x7f390c162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390bfff640 1 -- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f390c0b5420 msgr2=0x7f390c0b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390bfff640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f390c0b5420 0x7f390c0b7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390bfff640 1 -- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f390c0d0700 msgr2=0x7f390c162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390bfff640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f390c0d0700 0x7f390c162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390bfff640 1 -- 172.21.15.52:0/287058001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c0b3710 con 0x7f390c15f430 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f390bfff640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c15f430 0x7f390c147ce0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3904034040 tx=0x7f3904004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f38f3fff640 1 -- 172.21.15.52:0/287058001 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904036050 con 0x7f390c15f430 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f38f3fff640 1 -- 172.21.15.52:0/287058001 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3904042070 con 0x7f390c15f430 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c0b2950 con 0x7f390c15f430 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f38f3fff640 1 -- 172.21.15.52:0/287058001 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904009510 con 0x7f390c15f430 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.375+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c0b2f10 con 0x7f390c15f430 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.376+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c0695d0 con 0x7f390c15f430 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.377+0000 7f38f3fff640 1 -- 172.21.15.52:0/287058001 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f3904007bf0 con 0x7f390c15f430 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.377+0000 7f38f3fff640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f38f80764d0 0x7f38f8078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.377+0000 7f38f3fff640 1 -- 172.21.15.52:0/287058001 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f39040b88f0 con 0x7f390c15f430 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.378+0000 7f390b7fe640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f38f80764d0 0x7f38f8078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.378+0000 7f390b7fe640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f38f80764d0 0x7f38f8078990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f390c1f84c0 tx=0x7f38fc008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:22.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.382+0000 7f38f3fff640 1 -- 172.21.15.52:0/287058001 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f39040816d0 con 0x7f390c15f430 2024-05-11T01:13:22.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.599+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f390c0b8620 con 0x7f38f80764d0 2024-05-11T01:13:22.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.672+0000 7f38f3fff640 1 -- 172.21.15.52:0/287058001 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f38fc0108d0 con 0x7f38f80764d0 2024-05-11T01:13:22.672 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2024-05-11T01:13:22.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.674+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f38f80764d0 msgr2=0x7f38f8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.674+0000 7f3911b9d640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f38f80764d0 0x7f38f8078990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f390c1f84c0 tx=0x7f38fc008040 comp rx=0 tx=0).stop 2024-05-11T01:13:22.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c15f430 msgr2=0x7f390c147ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c15f430 0x7f390c147ce0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3904034040 tx=0x7f3904004a40 comp rx=0 tx=0).stop 2024-05-11T01:13:22.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 shutdown_connections 2024-05-11T01:13:22.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f38f80764d0 0x7f38f8078990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c15f430 0x7f390c147ce0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f390c0d0700 0x7f390c162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 --2- 172.21.15.52:0/287058001 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f390c0b5420 0x7f390c0b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 >> 172.21.15.52:0/287058001 conn(0x7f390c08ebb0 msgr2=0x7f390c06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:22.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 shutdown_connections 2024-05-11T01:13:22.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.675+0000 7f3911b9d640 1 -- 172.21.15.52:0/287058001 wait complete. 2024-05-11T01:13:22.685 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2024-05-11T01:13:22.685 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1m 2024-05-11T01:13:22.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.792+0000 7f170d659640 1 Processor -- start 2024-05-11T01:13:22.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.792+0000 7f170d659640 1 -- start start 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.793+0000 7f170d659640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f17080d3ef0 0x7f17080c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.793+0000 7f170d659640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1708163df0 con 0x7f170815f6d0 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.793+0000 7f170d659640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17080d00f0 con 0x7f170815f2f0 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.793+0000 7f170d659640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f17080d03d0 con 0x7f17080d3ef0 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.793+0000 7f17077fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f17080d3ef0 0x7f17080c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.793+0000 7f17067fc640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f170815f6d0 0x7f1708163070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55322/0 (socket says 172.21.15.52:55322) 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.793+0000 7f17067fc640 1 -- 172.21.15.52:0/1916577581 learned_addr learned my addr 172.21.15.52:0/1916577581 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.794+0000 7f17077fe640 1 -- 172.21.15.52:0/1916577581 >> v1:172.21.15.119:6789/0 conn(0x7f170815f2f0 legacy=0x7f17080d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.794+0000 7f17077fe640 1 -- 172.21.15.52:0/1916577581 >> v1:172.21.15.52:6789/0 conn(0x7f170815f6d0 legacy=0x7f1708163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.794+0000 7f17077fe640 1 -- 172.21.15.52:0/1916577581 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17080d06b0 con 0x7f17080d3ef0 2024-05-11T01:13:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.794+0000 7f1705ffb640 1 -- 172.21.15.52:0/1916577581 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3648535873 0 0) 0x7f1708163df0 con 0x7f170815f6d0 2024-05-11T01:13:22.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.794+0000 7f17077fe640 1 --2- 172.21.15.52:0/1916577581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 0x7f17080c8d50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17000097a0 tx=0x7f170002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d12b39f7952ea14 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:22.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.794+0000 7f1705ffb640 1 -- 172.21.15.52:0/1916577581 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700037070 con 0x7f17080d3ef0 2024-05-11T01:13:22.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.795+0000 7f1705ffb640 1 -- 172.21.15.52:0/1916577581 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1700032440 con 0x7f17080d3ef0 2024-05-11T01:13:22.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.795+0000 7f1705ffb640 1 -- 172.21.15.52:0/1916577581 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700032810 con 0x7f17080d3ef0 2024-05-11T01:13:22.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.795+0000 7f170d659640 1 -- 172.21.15.52:0/1916577581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 msgr2=0x7f17080c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.795+0000 7f170d659640 1 --2- 172.21.15.52:0/1916577581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 0x7f17080c8d50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17000097a0 tx=0x7f170002a230 comp rx=0 tx=0).stop 2024-05-11T01:13:22.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.795+0000 7f170d659640 1 -- 172.21.15.52:0/1916577581 shutdown_connections 2024-05-11T01:13:22.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.795+0000 7f170d659640 1 --2- 172.21.15.52:0/1916577581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 0x7f17080c8d50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.795+0000 7f170d659640 1 -- 172.21.15.52:0/1916577581 >> 172.21.15.52:0/1916577581 conn(0x7f170808ebc0 msgr2=0x7f170809e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:22.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.795+0000 7f170d659640 1 -- 172.21.15.52:0/1916577581 shutdown_connections 2024-05-11T01:13:22.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.795+0000 7f170d659640 1 -- 172.21.15.52:0/1916577581 wait complete. 2024-05-11T01:13:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.796+0000 7f170d659640 1 Processor -- start 2024-05-11T01:13:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.796+0000 7f170d659640 1 -- start start 2024-05-11T01:13:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.797+0000 7f170d659640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 0x7f1708153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.797+0000 7f170d659640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f170815f2f0 0x7f17081e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.797+0000 7f170d659640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f170815f6d0 0x7f17080cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.797+0000 7f170d659640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1708163900 con 0x7f17080d3ef0 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.797+0000 7f170d659640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1708163780 con 0x7f170815f6d0 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.797+0000 7f170d659640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f1708163a80 con 0x7f170815f2f0 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.797+0000 7f1706ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 0x7f1708153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.797+0000 7f1706ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 0x7f1708153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49986/0 (socket says 172.21.15.52:49986) 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.797+0000 7f1706ffd640 1 -- 172.21.15.52:0/626257455 learned_addr learned my addr 172.21.15.52:0/626257455 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f17077fe640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f170815f6d0 0x7f17080cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f1706ffd640 1 -- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f170815f2f0 msgr2=0x7f17081e96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f1706ffd640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f170815f2f0 0x7f17081e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f1706ffd640 1 -- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f170815f6d0 msgr2=0x7f17080cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f1706ffd640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f170815f6d0 0x7f17080cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f1706ffd640 1 -- 172.21.15.52:0/626257455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1708153a90 con 0x7f17080d3ef0 2024-05-11T01:13:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f1706ffd640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 0x7f1708153550 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f16f800c8a0 tx=0x7f16f800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f16ef7fe640 1 -- 172.21.15.52:0/626257455 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f8003b70 con 0x7f17080d3ef0 2024-05-11T01:13:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f16ef7fe640 1 -- 172.21.15.52:0/626257455 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f8003d10 con 0x7f17080d3ef0 2024-05-11T01:13:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1708161540 con 0x7f17080d3ef0 2024-05-11T01:13:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f16ef7fe640 1 -- 172.21.15.52:0/626257455 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f800aa10 con 0x7f17080d3ef0 2024-05-11T01:13:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.798+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1708161b00 con 0x7f17080d3ef0 2024-05-11T01:13:22.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.799+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1708069c00 con 0x7f17080d3ef0 2024-05-11T01:13:22.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.800+0000 7f16ef7fe640 1 -- 172.21.15.52:0/626257455 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f16f800abb0 con 0x7f17080d3ef0 2024-05-11T01:13:22.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.800+0000 7f16ef7fe640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16f40764d0 0x7f16f4078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:22.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.800+0000 7f16ef7fe640 1 -- 172.21.15.52:0/626257455 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f16f80979f0 con 0x7f17080d3ef0 2024-05-11T01:13:22.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.801+0000 7f17067fc640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16f40764d0 0x7f16f4078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:22.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.801+0000 7f17067fc640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16f40764d0 0x7f16f4078990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16fc000c00 tx=0x7f16fc0079c0 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:22.805+0000 7f16ef7fe640 1 -- 172.21.15.52:0/626257455 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f16f8060fd0 con 0x7f17080d3ef0 2024-05-11T01:13:23.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.024+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f17080d60f0 con 0x7f16f40764d0 2024-05-11T01:13:23.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.100+0000 7f16ef7fe640 1 -- 172.21.15.52:0/626257455 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f17080d60f0 con 0x7f16f40764d0 2024-05-11T01:13:23.099 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16f40764d0 msgr2=0x7f16f4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16f40764d0 0x7f16f4078990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16fc000c00 tx=0x7f16fc0079c0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 msgr2=0x7f1708153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 0x7f1708153550 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f16f800c8a0 tx=0x7f16f800cd60 comp rx=0 tx=0).stop 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 shutdown_connections 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f170815f6d0 0x7f17080cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f170815f2f0 0x7f17081e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f16f40764d0 0x7f16f4078990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 --2- 172.21.15.52:0/626257455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17080d3ef0 0x7f1708153550 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 >> 172.21.15.52:0/626257455 conn(0x7f170808ebc0 msgr2=0x7f17080c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:23.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 shutdown_connections 2024-05-11T01:13:23.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.102+0000 7f170d659640 1 -- 172.21.15.52:0/626257455 wait complete. 2024-05-11T01:13:23.112 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2024-05-11T01:13:23.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1m 2024-05-11T01:13:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.222+0000 7fb3d0272640 1 Processor -- start 2024-05-11T01:13:23.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.222+0000 7fb3d0272640 1 -- start start 2024-05-11T01:13:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3d0272640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3c80d1de0 0x7fb3c80d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3d0272640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3c80d2250 con 0x7fb3c815f430 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3d0272640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3c80cfc60 con 0x7fb3c80d1970 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3d0272640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c80d2a80 con 0x7fb3c80d1de0 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3ce80f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3c80d1de0 0x7fb3c80d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3ce80f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb3c80d1de0 0x7fb3c80d4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49992/0 (socket says 172.21.15.52:49992) 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3ce00e640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fb3c815f430 0x7fb3c80b4ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55330/0 (socket says 172.21.15.52:55330) 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3ce80f640 1 -- 172.21.15.52:0/1467578996 learned_addr learned my addr 172.21.15.52:0/1467578996 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3ce80f640 1 -- 172.21.15.52:0/1467578996 >> v1:172.21.15.119:6789/0 conn(0x7fb3c80d1970 legacy=0x7fb3c80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3ce80f640 1 -- 172.21.15.52:0/1467578996 >> v1:172.21.15.52:6789/0 conn(0x7fb3c815f430 legacy=0x7fb3c80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3ce80f640 1 -- 172.21.15.52:0/1467578996 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c8163120 con 0x7fb3c80d1de0 2024-05-11T01:13:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.223+0000 7fb3cd00c640 1 -- 172.21.15.52:0/1467578996 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638356282 0 0) 0x7fb3c80d2250 con 0x7fb3c815f430 2024-05-11T01:13:23.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.224+0000 7fb3ce80f640 1 --2- 172.21.15.52:0/1467578996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1de0 0x7fb3c80d4e00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009e50 tx=0x7fb3c002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cfb768cce629d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:23.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.224+0000 7fb3cd00c640 1 -- 172.21.15.52:0/1467578996 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0036070 con 0x7fb3c80d1de0 2024-05-11T01:13:23.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.224+0000 7fb3cd00c640 1 -- 172.21.15.52:0/1467578996 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0031600 con 0x7fb3c80d1de0 2024-05-11T01:13:23.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.224+0000 7fb3cd00c640 1 -- 172.21.15.52:0/1467578996 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c00319d0 con 0x7fb3c80d1de0 2024-05-11T01:13:23.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.225+0000 7fb3d0272640 1 -- 172.21.15.52:0/1467578996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1de0 msgr2=0x7fb3c80d4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.225+0000 7fb3d0272640 1 --2- 172.21.15.52:0/1467578996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1de0 0x7fb3c80d4e00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009e50 tx=0x7fb3c002a710 comp rx=0 tx=0).stop 2024-05-11T01:13:23.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.225+0000 7fb3d0272640 1 -- 172.21.15.52:0/1467578996 shutdown_connections 2024-05-11T01:13:23.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.225+0000 7fb3d0272640 1 --2- 172.21.15.52:0/1467578996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1de0 0x7fb3c80d4e00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.225+0000 7fb3d0272640 1 -- 172.21.15.52:0/1467578996 >> 172.21.15.52:0/1467578996 conn(0x7fb3c808ebb0 msgr2=0x7fb3c809e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:23.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.225+0000 7fb3d0272640 1 -- 172.21.15.52:0/1467578996 shutdown_connections 2024-05-11T01:13:23.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.225+0000 7fb3d0272640 1 -- 172.21.15.52:0/1467578996 wait complete. 2024-05-11T01:13:23.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.226+0000 7fb3d0272640 1 Processor -- start 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.226+0000 7fb3d0272640 1 -- start start 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3d0272640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1970 0x7fb3c80cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3d0272640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3c80d1de0 0x7fb3c815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3d0272640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c815f430 0x7fb3c815ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3d0272640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c80cff00 con 0x7fb3c80d1970 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3d0272640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c80d27a0 con 0x7fb3c815f430 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3d0272640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c80d01e0 con 0x7fb3c80d1de0 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce00e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1970 0x7fb3c80cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce00e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1970 0x7fb3c80cae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49996/0 (socket says 172.21.15.52:49996) 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce00e640 1 -- 172.21.15.52:0/2832091741 learned_addr learned my addr 172.21.15.52:0/2832091741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce80f640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c815f430 0x7fb3c815ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3cd80d640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3c80d1de0 0x7fb3c815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce00e640 1 -- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3c80d1de0 msgr2=0x7fb3c815fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce00e640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3c80d1de0 0x7fb3c815fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce00e640 1 -- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c815f430 msgr2=0x7fb3c815ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce00e640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c815f430 0x7fb3c815ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce00e640 1 -- 172.21.15.52:0/2832091741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c81620b0 con 0x7fb3c80d1970 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce80f640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c815f430 0x7fb3c815ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3cd80d640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3c80d1de0 0x7fb3c815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.227+0000 7fb3ce00e640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1970 0x7fb3c80cae90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400c490 tx=0x7fb3c400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.228+0000 7fb3beffd640 1 -- 172.21.15.52:0/2832091741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4018070 con 0x7fb3c80d1970 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.228+0000 7fb3beffd640 1 -- 172.21.15.52:0/2832091741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c4007500 con 0x7fb3c80d1970 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.228+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c8162320 con 0x7fb3c80d1970 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.228+0000 7fb3beffd640 1 -- 172.21.15.52:0/2832091741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4004a20 con 0x7fb3c80d1970 2024-05-11T01:13:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.228+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c80ca970 con 0x7fb3c80d1970 2024-05-11T01:13:23.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.229+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c80695d0 con 0x7fb3c80d1970 2024-05-11T01:13:23.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.229+0000 7fb3beffd640 1 -- 172.21.15.52:0/2832091741 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fb3c401b070 con 0x7fb3c80d1970 2024-05-11T01:13:23.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.230+0000 7fb3beffd640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fb3ac0764d0 0x7fb3ac078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.230+0000 7fb3beffd640 1 -- 172.21.15.52:0/2832091741 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7fb3c4096e60 con 0x7fb3c80d1970 2024-05-11T01:13:23.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.230+0000 7fb3cd80d640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fb3ac0764d0 0x7fb3ac078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.231+0000 7fb3cd80d640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fb3ac0764d0 0x7fb3ac078990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80b4fe0 tx=0x7fb3b8008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.234+0000 7fb3beffd640 1 -- 172.21.15.52:0/2832091741 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb3c40603a0 con 0x7fb3c80d1970 2024-05-11T01:13:23.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.451+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb3c80b5010 con 0x7fb3ac0764d0 2024-05-11T01:13:23.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.527+0000 7fb3beffd640 1 -- 172.21.15.52:0/2832091741 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb3c80b5010 con 0x7fb3ac0764d0 2024-05-11T01:13:23.526 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2024-05-11T01:13:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fb3ac0764d0 msgr2=0x7fb3ac078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fb3ac0764d0 0x7fb3ac078990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80b4fe0 tx=0x7fb3b8008040 comp rx=0 tx=0).stop 2024-05-11T01:13:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1970 msgr2=0x7fb3c80cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1970 0x7fb3c80cae90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400c490 tx=0x7fb3c400c950 comp rx=0 tx=0).stop 2024-05-11T01:13:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 shutdown_connections 2024-05-11T01:13:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c815f430 0x7fb3c815ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3c80d1de0 0x7fb3c815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fb3ac0764d0 0x7fb3ac078990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 --2- 172.21.15.52:0/2832091741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3c80d1970 0x7fb3c80cae90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.529+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 >> 172.21.15.52:0/2832091741 conn(0x7fb3c808ebb0 msgr2=0x7fb3c80d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:23.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.530+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 shutdown_connections 2024-05-11T01:13:23.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.530+0000 7fb3d0272640 1 -- 172.21.15.52:0/2832091741 wait complete. 2024-05-11T01:13:23.539 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2024-05-11T01:13:23.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:23.540 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_02_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:23.562 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:23.563 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:23.563 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_02_00_UTC', 'scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_00_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:23.563 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2024-05-11T01:13:23.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.709+0000 7f025c26d640 1 Processor -- start 2024-05-11T01:13:23.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.709+0000 7f025c26d640 1 -- start start 2024-05-11T01:13:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.710+0000 7f025c26d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f025415f2f0 0x7f02540c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.710+0000 7f025c26d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f025415f6d0 0x7f02540d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025c26d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f02540d3d20 0x7f0254163070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025c26d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f02540d4150 con 0x7f025415f2f0 2024-05-11T01:13:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025c26d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0254163df0 con 0x7f02540d3d20 2024-05-11T01:13:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025c26d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f02540d00f0 con 0x7f025415f6d0 2024-05-11T01:13:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025a009640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f025415f2f0 0x7f02540c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025a009640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f025415f2f0 0x7f02540c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50012/0 (socket says 172.21.15.52:50012) 2024-05-11T01:13:23.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025a009640 1 -- 172.21.15.52:0/3866259838 learned_addr learned my addr 172.21.15.52:0/3866259838 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025a80a640 1 --2- 172.21.15.52:0/3866259838 >> v2:172.21.15.119:3300/0 conn(0x7f02540d3d20 0x7f0254163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f0259808640 1 --2- 172.21.15.52:0/3866259838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f025415f6d0 0x7f02540d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025a009640 1 -- 172.21.15.52:0/3866259838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f025415f6d0 msgr2=0x7f02540d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025a009640 1 --2- 172.21.15.52:0/3866259838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f025415f6d0 0x7f02540d37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025a009640 1 -- 172.21.15.52:0/3866259838 >> v2:172.21.15.119:3300/0 conn(0x7f02540d3d20 msgr2=0x7f0254163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025a009640 1 --2- 172.21.15.52:0/3866259838 >> v2:172.21.15.119:3300/0 conn(0x7f02540d3d20 0x7f0254163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.711+0000 7f025a009640 1 -- 172.21.15.52:0/3866259838 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02540d03d0 con 0x7f025415f2f0 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.712+0000 7f025a009640 1 --2- 172.21.15.52:0/3866259838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f2f0 0x7f02540c8d50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f02540b6490 tx=0x7f025002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ad478267b1eb33 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.712+0000 7f0259007640 1 -- 172.21.15.52:0/3866259838 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250037070 con 0x7f025415f2f0 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.712+0000 7f0259007640 1 -- 172.21.15.52:0/3866259838 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0250032440 con 0x7f025415f2f0 2024-05-11T01:13:23.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.712+0000 7f0259007640 1 -- 172.21.15.52:0/3866259838 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250032810 con 0x7f025415f2f0 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.713+0000 7f025c26d640 1 -- 172.21.15.52:0/3866259838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f2f0 msgr2=0x7f02540c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.713+0000 7f025c26d640 1 --2- 172.21.15.52:0/3866259838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f2f0 0x7f02540c8d50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f02540b6490 tx=0x7f025002a1e0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.713+0000 7f025c26d640 1 -- 172.21.15.52:0/3866259838 shutdown_connections 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.713+0000 7f025c26d640 1 --2- 172.21.15.52:0/3866259838 >> v2:172.21.15.119:3300/0 conn(0x7f02540d3d20 0x7f0254163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.713+0000 7f025c26d640 1 --2- 172.21.15.52:0/3866259838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f025415f6d0 0x7f02540d37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.713+0000 7f025c26d640 1 --2- 172.21.15.52:0/3866259838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f2f0 0x7f02540c8d50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.713+0000 7f025c26d640 1 -- 172.21.15.52:0/3866259838 >> 172.21.15.52:0/3866259838 conn(0x7f025408ebc0 msgr2=0x7f025409e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.713+0000 7f025c26d640 1 -- 172.21.15.52:0/3866259838 shutdown_connections 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.713+0000 7f025c26d640 1 -- 172.21.15.52:0/3866259838 wait complete. 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.714+0000 7f025c26d640 1 Processor -- start 2024-05-11T01:13:23.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.714+0000 7f025c26d640 1 -- start start 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025c26d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540d3d20 0x7f025414b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025c26d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f025415f2f0 0x7f02540b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025c26d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f6d0 0x7f0254161d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025c26d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f02540cfe90 con 0x7f025415f6d0 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025c26d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f02540cfd10 con 0x7f025415f2f0 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025c26d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f02541635b0 con 0x7f02540d3d20 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a80a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f6d0 0x7f0254161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a80a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f6d0 0x7f0254161d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50028/0 (socket says 172.21.15.52:50028) 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a80a640 1 -- 172.21.15.52:0/1152907504 learned_addr learned my addr 172.21.15.52:0/1152907504 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f0259808640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f025415f2f0 0x7f02540b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a009640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540d3d20 0x7f025414b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a80a640 1 -- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540d3d20 msgr2=0x7f025414b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a80a640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540d3d20 0x7f025414b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a80a640 1 -- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f025415f2f0 msgr2=0x7f02540b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a80a640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f025415f2f0 0x7f02540b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a80a640 1 -- 172.21.15.52:0/1152907504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02541f6c10 con 0x7f025415f6d0 2024-05-11T01:13:23.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025a80a640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f6d0 0x7f0254161d10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f024c00efc0 tx=0x7f024c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:23.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f0246ffd640 1 -- 172.21.15.52:0/1152907504 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c018020 con 0x7f025415f6d0 2024-05-11T01:13:23.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f0246ffd640 1 -- 172.21.15.52:0/1152907504 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c014480 con 0x7f025415f6d0 2024-05-11T01:13:23.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02541f6ef0 con 0x7f025415f6d0 2024-05-11T01:13:23.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.715+0000 7f0246ffd640 1 -- 172.21.15.52:0/1152907504 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c014870 con 0x7f025415f6d0 2024-05-11T01:13:23.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.716+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02541f74b0 con 0x7f025415f6d0 2024-05-11T01:13:23.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.716+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0254069c00 con 0x7f025415f6d0 2024-05-11T01:13:23.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.717+0000 7f0246ffd640 1 -- 172.21.15.52:0/1152907504 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f024c005d10 con 0x7f025415f6d0 2024-05-11T01:13:23.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.718+0000 7f0246ffd640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f02380764d0 0x7f0238078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:23.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.718+0000 7f0246ffd640 1 -- 172.21.15.52:0/1152907504 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f024c0985b0 con 0x7f025415f6d0 2024-05-11T01:13:23.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.718+0000 7f025a009640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f02380764d0 0x7f0238078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:23.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.719+0000 7f025a009640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f02380764d0 0x7f0238078990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0250034000 tx=0x7f025002e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:23.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.722+0000 7f0246ffd640 1 -- 172.21.15.52:0/1152907504 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f024c022070 con 0x7f025415f6d0 2024-05-11T01:13:23.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.935+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02540d60f0 con 0x7f02380764d0 2024-05-11T01:13:23.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.944+0000 7f0246ffd640 1 -- 172.21.15.52:0/1152907504 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f02540d60f0 con 0x7f02380764d0 2024-05-11T01:13:23.943 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:23.943 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:01:58", "first": "2024-05-11T01:02:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2024-05-11T01:13:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f02380764d0 msgr2=0x7f0238078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f02380764d0 0x7f0238078990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0250034000 tx=0x7f025002e000 comp rx=0 tx=0).stop 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f6d0 msgr2=0x7f0254161d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f6d0 0x7f0254161d10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f024c00efc0 tx=0x7f024c00c490 comp rx=0 tx=0).stop 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 shutdown_connections 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f02380764d0 0x7f0238078990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f025415f6d0 0x7f0254161d10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f025415f2f0 0x7f02540b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 --2- 172.21.15.52:0/1152907504 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02540d3d20 0x7f025414b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 >> 172.21.15.52:0/1152907504 conn(0x7f025408ebc0 msgr2=0x7f02540681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 shutdown_connections 2024-05-11T01:13:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:23.946+0000 7f025c26d640 1 -- 172.21.15.52:0/1152907504 wait complete. 2024-05-11T01:13:23.956 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:01:58", "first": "2024-05-11T01:02:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2024-05-11T01:13:23.956 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:01:58', 'first': '2024-05-11T01:02:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:23.956 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2024-05-11T01:13:23.956 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2024-05-11T01:13:23.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:23.957 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_02_00_UTC 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:23.979 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:23.979 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:23.979 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_02_00_UTC', 'scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_00_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:23.979 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2024-05-11T01:13:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.124+0000 7f5be9124640 1 Processor -- start 2024-05-11T01:13:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.124+0000 7f5be9124640 1 -- start start 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.125+0000 7f5be9124640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be415fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.125+0000 7f5be9124640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5be406c480 0x7f5be406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.125+0000 7f5be9124640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be415c4b0 con 0x7f5be415c0d0 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.125+0000 7f5be9124640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5be406ce10 con 0x7f5be406c480 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.125+0000 7f5be9124640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5be4160ad0 con 0x7f5be406e880 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.125+0000 7f5bda59c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be415fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.125+0000 7f5bda59c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be415fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:50016/0 (socket says 172.21.15.52:50016) 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.125+0000 7f5bda59c640 1 -- 172.21.15.52:0/3395517209 learned_addr learned my addr 172.21.15.52:0/3395517209 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.125+0000 7f5be359e640 1 --2- 172.21.15.52:0/3395517209 >> v2:172.21.15.119:3300/0 conn(0x7f5be406c480 0x7f5be406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.126+0000 7f5bda59c640 1 -- 172.21.15.52:0/3395517209 >> v2:172.21.15.119:3300/0 conn(0x7f5be406c480 msgr2=0x7f5be406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.126+0000 7f5bda59c640 1 --2- 172.21.15.52:0/3395517209 >> v2:172.21.15.119:3300/0 conn(0x7f5be406c480 0x7f5be406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.126+0000 7f5bda59c640 1 -- 172.21.15.52:0/3395517209 >> v1:172.21.15.119:6789/0 conn(0x7f5be415c0d0 legacy=0x7f5be4072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.126+0000 7f5bda59c640 1 -- 172.21.15.52:0/3395517209 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be4160510 con 0x7f5be406e880 2024-05-11T01:13:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.126+0000 7f5be359e640 1 --2- 172.21.15.52:0/3395517209 >> v2:172.21.15.119:3300/0 conn(0x7f5be406c480 0x7f5be406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.126+0000 7f5bda59c640 1 --2- 172.21.15.52:0/3395517209 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be415fba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009870 tx=0x7f5bcc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2088f7367e18fded server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be259c640 1 -- 172.21.15.52:0/3395517209 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc037070 con 0x7f5be406e880 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be259c640 1 -- 172.21.15.52:0/3395517209 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc032440 con 0x7f5be406e880 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be259c640 1 -- 172.21.15.52:0/3395517209 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc032810 con 0x7f5be406e880 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be9124640 1 -- 172.21.15.52:0/3395517209 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 msgr2=0x7f5be415fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be9124640 1 --2- 172.21.15.52:0/3395517209 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be415fba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009870 tx=0x7f5bcc02a1c0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be9124640 1 -- 172.21.15.52:0/3395517209 shutdown_connections 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be9124640 1 --2- 172.21.15.52:0/3395517209 >> v2:172.21.15.119:3300/0 conn(0x7f5be406c480 0x7f5be406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be9124640 1 --2- 172.21.15.52:0/3395517209 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be415fba0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be9124640 1 -- 172.21.15.52:0/3395517209 >> 172.21.15.52:0/3395517209 conn(0x7f5be40cd830 msgr2=0x7f5be40cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be9124640 1 -- 172.21.15.52:0/3395517209 shutdown_connections 2024-05-11T01:13:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.127+0000 7f5be9124640 1 -- 172.21.15.52:0/3395517209 wait complete. 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.128+0000 7f5be9124640 1 Processor -- start 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.129+0000 7f5be9124640 1 -- start start 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.129+0000 7f5be9124640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be406c480 0x7f5be4150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.129+0000 7f5be9124640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be41e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.129+0000 7f5be9124640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5be415c0d0 0x7f5be415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.129+0000 7f5be9124640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5be40cc490 con 0x7f5be406c480 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.129+0000 7f5be9124640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5be40cc1b0 con 0x7f5be415c0d0 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.129+0000 7f5be9124640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5be41600e0 con 0x7f5be406e880 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.129+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be406c480 0x7f5be4150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be406c480 0x7f5be4150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50044/0 (socket says 172.21.15.52:50044) 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be2d9d640 1 -- 172.21.15.52:0/936276508 learned_addr learned my addr 172.21.15.52:0/936276508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be359e640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5be415c0d0 0x7f5be415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5bda59c640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be41e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be2d9d640 1 -- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 msgr2=0x7f5be41e63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be2d9d640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be41e63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be2d9d640 1 -- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5be415c0d0 msgr2=0x7f5be415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be2d9d640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5be415c0d0 0x7f5be415a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be2d9d640 1 -- 172.21.15.52:0/936276508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be41e6d80 con 0x7f5be406c480 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be2d9d640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be406c480 0x7f5be4150360 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400c3a0 tx=0x7f5bd400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5bdbfff640 1 -- 172.21.15.52:0/936276508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4017070 con 0x7f5be406c480 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5bdbfff640 1 -- 172.21.15.52:0/936276508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd40052d0 con 0x7f5be406c480 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5bdbfff640 1 -- 172.21.15.52:0/936276508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4003830 con 0x7f5be406c480 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be41e7060 con 0x7f5be406c480 2024-05-11T01:13:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.130+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be4159a50 con 0x7f5be406c480 2024-05-11T01:13:24.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.131+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be40695d0 con 0x7f5be406c480 2024-05-11T01:13:24.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.132+0000 7f5bdbfff640 1 -- 172.21.15.52:0/936276508 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f5bd401a070 con 0x7f5be406c480 2024-05-11T01:13:24.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.132+0000 7f5bdbfff640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5bc40764d0 0x7f5bc4078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.132+0000 7f5bdbfff640 1 -- 172.21.15.52:0/936276508 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f5bd4096e90 con 0x7f5be406c480 2024-05-11T01:13:24.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.133+0000 7f5bda59c640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5bc40764d0 0x7f5bc4078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.133+0000 7f5bda59c640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5bc40764d0 0x7f5bc4078990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc034000 tx=0x7f5bcc02e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:24.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.137+0000 7f5bdbfff640 1 -- 172.21.15.52:0/936276508 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5bd40603d0 con 0x7f5be406c480 2024-05-11T01:13:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.351+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5be40c7e60 con 0x7f5bc40764d0 2024-05-11T01:13:24.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.360+0000 7f5bdbfff640 1 -- 172.21.15.52:0/936276508 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f5bcc032740 con 0x7f5bc40764d0 2024-05-11T01:13:24.359 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:24.359 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:01:59", "first": "2024-05-11T01:02:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2024-05-11T01:13:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5bc40764d0 msgr2=0x7f5bc4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5bc40764d0 0x7f5bc4078990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc034000 tx=0x7f5bcc02e000 comp rx=0 tx=0).stop 2024-05-11T01:13:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be406c480 msgr2=0x7f5be4150360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be406c480 0x7f5be4150360 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400c3a0 tx=0x7f5bd400c860 comp rx=0 tx=0).stop 2024-05-11T01:13:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 shutdown_connections 2024-05-11T01:13:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5be415c0d0 0x7f5be415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5be406e880 0x7f5be41e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5bc40764d0 0x7f5bc4078990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 --2- 172.21.15.52:0/936276508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be406c480 0x7f5be4150360 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 >> 172.21.15.52:0/936276508 conn(0x7f5be40cd830 msgr2=0x7f5be40cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 shutdown_connections 2024-05-11T01:13:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.362+0000 7f5be9124640 1 -- 172.21.15.52:0/936276508 wait complete. 2024-05-11T01:13:24.372 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:01:59", "first": "2024-05-11T01:02:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2024-05-11T01:13:24.372 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:01:59', 'first': '2024-05-11T01:02:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:24.372 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2024-05-11T01:13:24.372 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2024-05-11T01:13:24.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:24.372 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2024-05-11T01:13:24.394 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_02_00_UTC 2024-05-11T01:13:24.394 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:24.394 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:24.394 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:24.394 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:24.394 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:24.394 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:24.395 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:24.395 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:24.395 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:24.395 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:24.395 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:24.395 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_02_00_UTC', 'scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_00_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:24.395 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2024-05-11T01:13:24.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.540+0000 7f9aa3b7f640 1 Processor -- start 2024-05-11T01:13:24.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.541+0000 7f9aa3b7f640 1 -- start start 2024-05-11T01:13:24.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.541+0000 7f9aa3b7f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9a9c0cd070 0x7f9a9c06e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.541+0000 7f9aa3b7f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a9c0b0120 con 0x7f9a9c15f300 2024-05-11T01:13:24.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.541+0000 7f9aa3b7f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a9c0b0400 con 0x7f9a9c15fe80 2024-05-11T01:13:24.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.541+0000 7f9aa3b7f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f9a9c06ebd0 con 0x7f9a9c0cd070 2024-05-11T01:13:24.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.541+0000 7f9aa191b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9a9c0cd070 0x7f9a9c06e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.541+0000 7f9aa111a640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f9a9c15f300 0x7f9a9c15f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55346/0 (socket says 172.21.15.52:55346) 2024-05-11T01:13:24.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.542+0000 7f9aa111a640 1 -- 172.21.15.52:0/466863119 learned_addr learned my addr 172.21.15.52:0/466863119 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:24.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.542+0000 7f9aa191b640 1 --2- 172.21.15.52:0/466863119 >> v2:172.21.15.52:3300/0 conn(0x7f9a9c0cd070 0x7f9a9c06e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50058/0 (socket says 172.21.15.52:50058) 2024-05-11T01:13:24.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.542+0000 7f9aa191b640 1 -- 172.21.15.52:0/466863119 >> v1:172.21.15.119:6789/0 conn(0x7f9a9c15fe80 legacy=0x7f9a9c0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:24.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.542+0000 7f9aa191b640 1 -- 172.21.15.52:0/466863119 >> v1:172.21.15.52:6789/0 conn(0x7f9a9c15f300 legacy=0x7f9a9c15f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.542+0000 7f9aa0919640 1 -- 172.21.15.52:0/466863119 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993314993 0 0) 0x7f9a9c0b0120 con 0x7f9a9c15f300 2024-05-11T01:13:24.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.542+0000 7f9aa191b640 1 -- 172.21.15.52:0/466863119 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c06f0f0 con 0x7f9a9c0cd070 2024-05-11T01:13:24.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa191b640 1 --2- 172.21.15.52:0/466863119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c0cd070 0x7f9a9c06e1c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9a980097a0 tx=0x7f9a9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6be80388eb16698 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:24.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa0919640 1 -- 172.21.15.52:0/466863119 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98037070 con 0x7f9a9c0cd070 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa0919640 1 -- 172.21.15.52:0/466863119 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a98032440 con 0x7f9a9c0cd070 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa0919640 1 -- 172.21.15.52:0/466863119 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98032810 con 0x7f9a9c0cd070 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/466863119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c0cd070 msgr2=0x7f9a9c06e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa3b7f640 1 --2- 172.21.15.52:0/466863119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c0cd070 0x7f9a9c06e1c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9a980097a0 tx=0x7f9a9802a210 comp rx=0 tx=0).stop 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/466863119 shutdown_connections 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa3b7f640 1 --2- 172.21.15.52:0/466863119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c0cd070 0x7f9a9c06e1c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/466863119 >> 172.21.15.52:0/466863119 conn(0x7f9a9c0cd830 msgr2=0x7f9a9c0cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/466863119 shutdown_connections 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.543+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/466863119 wait complete. 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.545+0000 7f9aa3b7f640 1 Processor -- start 2024-05-11T01:13:24.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.545+0000 7f9aa3b7f640 1 -- start start 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.545+0000 7f9aa3b7f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c0cd070 0x7f9a9c0c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.545+0000 7f9aa3b7f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a9c15f300 0x7f9a9c09db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.545+0000 7f9aa3b7f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c15fe80 0x7f9a9c072060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.545+0000 7f9aa3b7f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c06c600 con 0x7f9a9c15fe80 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.545+0000 7f9aa3b7f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c06c480 con 0x7f9a9c0cd070 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.545+0000 7f9aa3b7f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9c06c780 con 0x7f9a9c15f300 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa211c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c15fe80 0x7f9a9c072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa211c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c15fe80 0x7f9a9c072060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50072/0 (socket says 172.21.15.52:50072) 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa211c640 1 -- 172.21.15.52:0/1926120860 learned_addr learned my addr 172.21.15.52:0/1926120860 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa191b640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c0cd070 0x7f9a9c0c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa111a640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a9c15f300 0x7f9a9c09db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa211c640 1 -- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a9c15f300 msgr2=0x7f9a9c09db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa211c640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a9c15f300 0x7f9a9c09db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa211c640 1 -- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c0cd070 msgr2=0x7f9a9c0c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa211c640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c0cd070 0x7f9a9c0c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa211c640 1 -- 172.21.15.52:0/1926120860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c09e420 con 0x7f9a9c15fe80 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa191b640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c0cd070 0x7f9a9c0c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa211c640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c15fe80 0x7f9a9c072060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400c7b0 tx=0x7f9a9400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9a927fc640 1 -- 172.21.15.52:0/1926120860 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94004b60 con 0x7f9a9c15fe80 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9a927fc640 1 -- 172.21.15.52:0/1926120860 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a94004d00 con 0x7f9a9c15fe80 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9c09e700 con 0x7f9a9c15fe80 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9a927fc640 1 -- 172.21.15.52:0/1926120860 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400aa90 con 0x7f9a9c15fe80 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.546+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9c09e080 con 0x7f9a9c15fe80 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.547+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a9c0695d0 con 0x7f9a9c15fe80 2024-05-11T01:13:24.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.548+0000 7f9a927fc640 1 -- 172.21.15.52:0/1926120860 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f9a9400ac30 con 0x7f9a9c15fe80 2024-05-11T01:13:24.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.548+0000 7f9a927fc640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9a800764d0 0x7f9a80078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.549+0000 7f9a927fc640 1 -- 172.21.15.52:0/1926120860 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f9a94097d40 con 0x7f9a9c15fe80 2024-05-11T01:13:24.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.549+0000 7f9aa191b640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9a800764d0 0x7f9a80078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.549+0000 7f9aa191b640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9a800764d0 0x7f9a80078990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9a98034000 tx=0x7f9a9802e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:24.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.553+0000 7f9a927fc640 1 -- 172.21.15.52:0/1926120860 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9a94020070 con 0x7f9a9c15fe80 2024-05-11T01:13:24.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.765+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a9c0b6d30 con 0x7f9a800764d0 2024-05-11T01:13:24.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.774+0000 7f9a927fc640 1 -- 172.21.15.52:0/1926120860 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f9a9c0b6d30 con 0x7f9a800764d0 2024-05-11T01:13:24.773 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:24.773 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:01:59", "first": "2024-05-11T01:02:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2024-05-11T01:13:24.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9a800764d0 msgr2=0x7f9a80078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9a800764d0 0x7f9a80078990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9a98034000 tx=0x7f9a9802e000 comp rx=0 tx=0).stop 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c15fe80 msgr2=0x7f9a9c072060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c15fe80 0x7f9a9c072060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400c7b0 tx=0x7f9a9400cc70 comp rx=0 tx=0).stop 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 shutdown_connections 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f9a800764d0 0x7f9a80078990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a9c15fe80 0x7f9a9c072060 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9a9c15f300 0x7f9a9c09db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 --2- 172.21.15.52:0/1926120860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c0cd070 0x7f9a9c0c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 >> 172.21.15.52:0/1926120860 conn(0x7f9a9c0cd830 msgr2=0x7f9a9c0b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 shutdown_connections 2024-05-11T01:13:24.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.776+0000 7f9aa3b7f640 1 -- 172.21.15.52:0/1926120860 wait complete. 2024-05-11T01:13:24.786 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:01:59", "first": "2024-05-11T01:02:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2024-05-11T01:13:24.786 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:01:59', 'first': '2024-05-11T01:02:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:24.786 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2024-05-11T01:13:24.786 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2024-05-11T01:13:24.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:24.786 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2024-05-11T01:13:24.807 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:24.807 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_01_UTC 2024-05-11T01:13:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:24.808 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:24.808 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_01_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:24.808 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2024-05-11T01:13:24.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.954+0000 7f4f8099d640 1 Processor -- start 2024-05-11T01:13:24.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.954+0000 7f4f8099d640 1 -- start start 2024-05-11T01:13:24.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f8099d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4f7c06e880 0x7f4f7c15fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f8099d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f7c06c480 0x7f4f7c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f8099d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c15c4b0 con 0x7f4f7c15c0d0 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f8099d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f4f7c06ce10 con 0x7f4f7c06e880 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f8099d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c160ad0 con 0x7f4f7c06c480 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f79d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4f7c06e880 0x7f4f7c15fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f79d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f4f7c06e880 0x7f4f7c15fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50082/0 (socket says 172.21.15.52:50082) 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f79d9b640 1 -- 172.21.15.52:0/116767112 learned_addr learned my addr 172.21.15.52:0/116767112 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f7ad9d640 1 --2- 172.21.15.52:0/116767112 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f7c06c480 0x7f4f7c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f79d9b640 1 -- 172.21.15.52:0/116767112 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f7c06c480 msgr2=0x7f4f7c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f79d9b640 1 --2- 172.21.15.52:0/116767112 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f7c06c480 0x7f4f7c06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f79d9b640 1 -- 172.21.15.52:0/116767112 >> v1:172.21.15.119:6789/0 conn(0x7f4f7c15c0d0 legacy=0x7f4f7c072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.955+0000 7f4f79d9b640 1 -- 172.21.15.52:0/116767112 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c160510 con 0x7f4f7c06e880 2024-05-11T01:13:24.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.956+0000 7f4f79d9b640 1 --2- 172.21.15.52:0/116767112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 0x7f4f7c15fba0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009870 tx=0x7f4f7002a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d1f7042a191e46 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:24.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.956+0000 7f4f7959a640 1 -- 172.21.15.52:0/116767112 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70037070 con 0x7f4f7c06e880 2024-05-11T01:13:24.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.956+0000 7f4f7959a640 1 -- 172.21.15.52:0/116767112 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f70032440 con 0x7f4f7c06e880 2024-05-11T01:13:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.956+0000 7f4f7959a640 1 -- 172.21.15.52:0/116767112 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70032810 con 0x7f4f7c06e880 2024-05-11T01:13:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.957+0000 7f4f8099d640 1 -- 172.21.15.52:0/116767112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 msgr2=0x7f4f7c15fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.957+0000 7f4f8099d640 1 --2- 172.21.15.52:0/116767112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 0x7f4f7c15fba0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009870 tx=0x7f4f7002a1b0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.957+0000 7f4f8099d640 1 -- 172.21.15.52:0/116767112 shutdown_connections 2024-05-11T01:13:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.957+0000 7f4f8099d640 1 --2- 172.21.15.52:0/116767112 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f7c06c480 0x7f4f7c06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.957+0000 7f4f8099d640 1 --2- 172.21.15.52:0/116767112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 0x7f4f7c15fba0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.957+0000 7f4f8099d640 1 -- 172.21.15.52:0/116767112 >> 172.21.15.52:0/116767112 conn(0x7f4f7c0cd830 msgr2=0x7f4f7c0cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.957+0000 7f4f8099d640 1 -- 172.21.15.52:0/116767112 shutdown_connections 2024-05-11T01:13:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.957+0000 7f4f8099d640 1 -- 172.21.15.52:0/116767112 wait complete. 2024-05-11T01:13:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.958+0000 7f4f8099d640 1 Processor -- start 2024-05-11T01:13:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.958+0000 7f4f8099d640 1 -- start start 2024-05-11T01:13:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f8099d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f7c06c480 0x7f4f7c0af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f8099d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 0x7f4f7c09ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f8099d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f7c15c0d0 0x7f4f7c09d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f8099d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c0cc490 con 0x7f4f7c06e880 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f8099d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c0cc1b0 con 0x7f4f7c06c480 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f8099d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c1600e0 con 0x7f4f7c15c0d0 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f79d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 0x7f4f7c09ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f79d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 0x7f4f7c09ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50086/0 (socket says 172.21.15.52:50086) 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f79d9b640 1 -- 172.21.15.52:0/3235220451 learned_addr learned my addr 172.21.15.52:0/3235220451 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f79d9b640 1 -- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f7c15c0d0 msgr2=0x7f4f7c09d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f79d9b640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f7c15c0d0 0x7f4f7c09d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f79d9b640 1 -- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f7c06c480 msgr2=0x7f4f7c0af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f79d9b640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f7c06c480 0x7f4f7c0af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.959+0000 7f4f79d9b640 1 -- 172.21.15.52:0/3235220451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c09da70 con 0x7f4f7c06e880 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.960+0000 7f4f79d9b640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 0x7f4f7c09ccf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4f7002a1f0 tx=0x7f4f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.960+0000 7f4f6affd640 1 -- 172.21.15.52:0/3235220451 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70041020 con 0x7f4f7c06e880 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.960+0000 7f4f6affd640 1 -- 172.21.15.52:0/3235220451 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f700306c0 con 0x7f4f7c06e880 2024-05-11T01:13:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.960+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7c09dd50 con 0x7f4f7c06e880 2024-05-11T01:13:24.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.960+0000 7f4f6affd640 1 -- 172.21.15.52:0/3235220451 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70037040 con 0x7f4f7c06e880 2024-05-11T01:13:24.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.960+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7c09e310 con 0x7f4f7c06e880 2024-05-11T01:13:24.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.961+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7c0695d0 con 0x7f4f7c06e880 2024-05-11T01:13:24.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.962+0000 7f4f6affd640 1 -- 172.21.15.52:0/3235220451 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f4f70044020 con 0x7f4f7c06e880 2024-05-11T01:13:24.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.962+0000 7f4f6affd640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4f5c0764d0 0x7f4f5c078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:24.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.962+0000 7f4f6affd640 1 -- 172.21.15.52:0/3235220451 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f4f700b8290 con 0x7f4f7c06e880 2024-05-11T01:13:24.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.962+0000 7f4f7a59c640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4f5c0764d0 0x7f4f5c078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:24.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.963+0000 7f4f7a59c640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4f5c0764d0 0x7f4f5c078990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c000f60 tx=0x7f4f6c0079b0 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:24.966+0000 7f4f6affd640 1 -- 172.21.15.52:0/3235220451 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4f7004b030 con 0x7f4f7c06e880 2024-05-11T01:13:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.178+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f7c0d01b0 con 0x7f4f5c0764d0 2024-05-11T01:13:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.186+0000 7f4f6affd640 1 -- 172.21.15.52:0/3235220451 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f4f7c0d01b0 con 0x7f4f5c0764d0 2024-05-11T01:13:25.185 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:25.186 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:00", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4f5c0764d0 msgr2=0x7f4f5c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4f5c0764d0 0x7f4f5c078990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c000f60 tx=0x7f4f6c0079b0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 msgr2=0x7f4f7c09ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 0x7f4f7c09ccf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4f7002a1f0 tx=0x7f4f700046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 shutdown_connections 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4f7c15c0d0 0x7f4f7c09d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4f5c0764d0 0x7f4f5c078990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f7c06e880 0x7f4f7c09ccf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 --2- 172.21.15.52:0/3235220451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f7c06c480 0x7f4f7c0af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.188+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 >> 172.21.15.52:0/3235220451 conn(0x7f4f7c0cd830 msgr2=0x7f4f7c0729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.189+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 shutdown_connections 2024-05-11T01:13:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.189+0000 7f4f8099d640 1 -- 172.21.15.52:0/3235220451 wait complete. 2024-05-11T01:13:25.198 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:00", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:25.198 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:02:00', 'first': '2024-05-11T01:03:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:25.198 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-05-11T01:13:25.199 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-05-11T01:13:25.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:25.199 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2024-05-11T01:13:25.220 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:25.220 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:25.220 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:25.220 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:25.220 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:25.220 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:25.220 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:25.220 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:25.220 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:25.221 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:25.221 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:25.221 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_00_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:25.221 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2024-05-11T01:13:25.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.367+0000 7fe4dfd14640 1 Processor -- start 2024-05-11T01:13:25.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.367+0000 7fe4dfd14640 1 -- start start 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.368+0000 7fe4dfd14640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d8162990 0x7fe4d80d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.368+0000 7fe4dfd14640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d80d3390 con 0x7fe4d80d0740 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.368+0000 7fe4dfd14640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d8162d70 con 0x7fe4d80d3d60 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.368+0000 7fe4dfd14640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d80b7f50 con 0x7fe4d8162990 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.368+0000 7fe4dd2af640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fe4d80d0740 0x7fe4d80d3650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55358/0 (socket says 172.21.15.52:55358) 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.368+0000 7fe4dd2af640 1 -- 172.21.15.52:0/3167965016 learned_addr learned my addr 172.21.15.52:0/3167965016 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.368+0000 7fe4ddab0640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d8162990 0x7fe4d80d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4262291689 0 0) 0x7fe4d80d3390 con 0x7fe4d80d0740 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4c0003520 con 0x7fe4d80d0740 2024-05-11T01:13:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066907995 0 0) 0x7fe4d8162d70 con 0x7fe4d80d3d60 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4d80d3390 con 0x7fe4d80d3d60 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2164091967 0 0) 0x7fe4c0003520 con 0x7fe4d80d0740 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d8162990 msgr2=0x7fe4d80d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d8162990 0x7fe4d80d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 >> v1:172.21.15.119:6789/0 conn(0x7fe4d80d3d60 legacy=0x7fe4d80b89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d80d0bc0 con 0x7fe4d80d0740 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fe4cc0016f0 con 0x7fe4d80d0740 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 >> v1:172.21.15.52:6789/0 conn(0x7fe4d80d0740 legacy=0x7fe4d80d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c0003c10 0x7fe4c0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d8162990 0x7fe4c00052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d80b7f50 con 0x7fe4c0004800 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c000bbc0 con 0x7fe4c0003c10 2024-05-11T01:13:25.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c000bdc0 con 0x7fe4d8162990 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dd2af640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4ddab0640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c0003c10 0x7fe4c0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4de2b1640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d8162990 0x7fe4c00052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dd2af640 1 -- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d8162990 msgr2=0x7fe4c00052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dd2af640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d8162990 0x7fe4c00052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dd2af640 1 -- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c0003c10 msgr2=0x7fe4c0004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dd2af640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c0003c10 0x7fe4c0004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.369+0000 7fe4dd2af640 1 -- 172.21.15.52:0/3167965016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c0003520 con 0x7fe4c0004800 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.370+0000 7fe4dd2af640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4c0004c80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c190 tx=0x7fe4c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eed9b8067430c34 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.370+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc007ce0 con 0x7fe4c0004800 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.370+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc005af0 con 0x7fe4c0004800 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.370+0000 7fe4dcaae640 1 -- 172.21.15.52:0/3167965016 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc007070 con 0x7fe4c0004800 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.371+0000 7fe4dfd14640 1 -- 172.21.15.52:0/3167965016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 msgr2=0x7fe4c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.371+0000 7fe4dfd14640 1 --2- 172.21.15.52:0/3167965016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4c0004c80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c190 tx=0x7fe4c0005830 comp rx=0 tx=0).stop 2024-05-11T01:13:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.371+0000 7fe4de2b1640 1 -- 172.21.15.52:0/3167965016 reap_dead start 2024-05-11T01:13:25.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.371+0000 7fe4dfd14640 1 -- 172.21.15.52:0/3167965016 shutdown_connections 2024-05-11T01:13:25.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.371+0000 7fe4dfd14640 1 -- 172.21.15.52:0/3167965016 >> 172.21.15.52:0/3167965016 conn(0x7fe4d809e490 msgr2=0x7fe4d80d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:25.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.371+0000 7fe4dfd14640 1 -- 172.21.15.52:0/3167965016 shutdown_connections 2024-05-11T01:13:25.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.371+0000 7fe4dfd14640 1 -- 172.21.15.52:0/3167965016 wait complete. 2024-05-11T01:13:25.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.372+0000 7fe4dfd14640 1 Processor -- start 2024-05-11T01:13:25.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.372+0000 7fe4dfd14640 1 -- start start 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4dfd14640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4d8152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4dfd14640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d809ee00 0x7fe4d8161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4dfd14640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d81f52c0 0x7fe4d81f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4dfd14640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d80b81a0 con 0x7fe4c0004800 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4dfd14640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d80d5d40 con 0x7fe4d81f52c0 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4dfd14640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d80b8480 con 0x7fe4d809ee00 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4ddab0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4d8152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4ddab0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4d8152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50102/0 (socket says 172.21.15.52:50102) 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4ddab0640 1 -- 172.21.15.52:0/1229286253 learned_addr learned my addr 172.21.15.52:0/1229286253 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4de2b1640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d81f52c0 0x7fe4d81f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4ddab0640 1 -- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d809ee00 msgr2=0x7fe4d8161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4dd2af640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d809ee00 0x7fe4d8161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4ddab0640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d809ee00 0x7fe4d8161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4ddab0640 1 -- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d81f52c0 msgr2=0x7fe4d81f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4ddab0640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d81f52c0 0x7fe4d81f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4ddab0640 1 -- 172.21.15.52:0/1229286253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d8161690 con 0x7fe4c0004800 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4de2b1640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d81f52c0 0x7fe4d81f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.373+0000 7fe4ddab0640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4d8152440 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8011e30 tx=0x7fe4c800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.374+0000 7fe4d67fc640 1 -- 172.21.15.52:0/1229286253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8015630 con 0x7fe4c0004800 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.374+0000 7fe4d67fc640 1 -- 172.21.15.52:0/1229286253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c80157d0 con 0x7fe4c0004800 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.374+0000 7fe4d67fc640 1 -- 172.21.15.52:0/1229286253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8015ba0 con 0x7fe4c0004800 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.374+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d809f3a0 con 0x7fe4c0004800 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.374+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d809f960 con 0x7fe4c0004800 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.375+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d80695d0 con 0x7fe4c0004800 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.375+0000 7fe4d67fc640 1 -- 172.21.15.52:0/1229286253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fe4c801d400 con 0x7fe4c0004800 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.376+0000 7fe4d67fc640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fe4c40764d0 0x7fe4c4078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.376+0000 7fe4d67fc640 1 -- 172.21.15.52:0/1229286253 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7fe4c8098b70 con 0x7fe4c0004800 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.376+0000 7fe4dd2af640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fe4c40764d0 0x7fe4c4078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.377+0000 7fe4dd2af640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fe4c40764d0 0x7fe4c4078990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc032000 tx=0x7fe4cc02d000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:25.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.380+0000 7fe4d67fc640 1 -- 172.21.15.52:0/1229286253 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe4c8019030 con 0x7fe4c0004800 2024-05-11T01:13:25.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.600+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4d806d680 con 0x7fe4c40764d0 2024-05-11T01:13:25.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.608+0000 7fe4d67fc640 1 -- 172.21.15.52:0/1229286253 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7fe4d806d680 con 0x7fe4c40764d0 2024-05-11T01:13:25.608 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:25.608 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:00", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:25.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.610+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fe4c40764d0 msgr2=0x7fe4c4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.610+0000 7fe4dfd14640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fe4c40764d0 0x7fe4c4078990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc032000 tx=0x7fe4cc02d000 comp rx=0 tx=0).stop 2024-05-11T01:13:25.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.610+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 msgr2=0x7fe4d8152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.610+0000 7fe4dfd14640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4d8152440 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8011e30 tx=0x7fe4c800f580 comp rx=0 tx=0).stop 2024-05-11T01:13:25.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.610+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 shutdown_connections 2024-05-11T01:13:25.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.610+0000 7fe4dfd14640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4d81f52c0 0x7fe4d81f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.610+0000 7fe4dfd14640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4d809ee00 0x7fe4d8161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.611+0000 7fe4dfd14640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fe4c40764d0 0x7fe4c4078990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.611+0000 7fe4dfd14640 1 --2- 172.21.15.52:0/1229286253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c0004800 0x7fe4d8152440 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.611+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 >> 172.21.15.52:0/1229286253 conn(0x7fe4d809e490 msgr2=0x7fe4d80b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:25.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.611+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 shutdown_connections 2024-05-11T01:13:25.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.611+0000 7fe4dfd14640 1 -- 172.21.15.52:0/1229286253 wait complete. 2024-05-11T01:13:25.620 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:00", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:25.620 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:02:00', 'first': '2024-05-11T01:03:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:25.621 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-05-11T01:13:25.621 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-05-11T01:13:25.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:25.621 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2024-05-11T01:13:25.641 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:25.642 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:25.642 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_00_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:25.643 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2024-05-11T01:13:25.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.788+0000 7f5cb4045640 1 Processor -- start 2024-05-11T01:13:25.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.788+0000 7f5cb4045640 1 -- start start 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb4045640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac06d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb4045640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5cac09c130 0x7f5cac0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb4045640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cac06e140 con 0x7f5cac0b4320 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb4045640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5cac06dd80 con 0x7f5cac09c130 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb4045640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5cac06e420 con 0x7f5cac0d2b40 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb15e0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac06d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb25e2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5cac09c130 0x7f5cac0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb15e0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac06d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:50068/0 (socket says 172.21.15.52:50068) 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb15e0640 1 -- 172.21.15.52:0/3617165576 learned_addr learned my addr 172.21.15.52:0/3617165576 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:25.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.789+0000 7f5cb1de1640 1 --1- 172.21.15.52:0/3617165576 >> v1:172.21.15.119:6789/0 conn(0x7f5cac0b4320 0x7f5cac09ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:39316/0 (socket says 172.21.15.52:39316) 2024-05-11T01:13:25.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.790+0000 7f5cb15e0640 1 -- 172.21.15.52:0/3617165576 >> v2:172.21.15.119:3300/0 conn(0x7f5cac09c130 msgr2=0x7f5cac0b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:25.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.790+0000 7f5cb15e0640 1 --2- 172.21.15.52:0/3617165576 >> v2:172.21.15.119:3300/0 conn(0x7f5cac09c130 0x7f5cac0b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.790+0000 7f5cb15e0640 1 -- 172.21.15.52:0/3617165576 >> v1:172.21.15.119:6789/0 conn(0x7f5cac0b4320 legacy=0x7f5cac09ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.790+0000 7f5cb15e0640 1 -- 172.21.15.52:0/3617165576 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac0d1e60 con 0x7f5cac0d2b40 2024-05-11T01:13:25.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.790+0000 7f5cb0ddf640 1 -- 172.21.15.52:0/3617165576 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 8840727 0 0) 0x7f5cac06e140 con 0x7f5cac0b4320 2024-05-11T01:13:25.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.790+0000 7f5cb15e0640 1 --2- 172.21.15.52:0/3617165576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac06d7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009870 tx=0x7f5c9c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f61f63a135a679 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb0ddf640 1 -- 172.21.15.52:0/3617165576 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c037070 con 0x7f5cac0d2b40 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb0ddf640 1 -- 172.21.15.52:0/3617165576 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9c032440 con 0x7f5cac0d2b40 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb0ddf640 1 -- 172.21.15.52:0/3617165576 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c032810 con 0x7f5cac0d2b40 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb4045640 1 -- 172.21.15.52:0/3617165576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 msgr2=0x7f5cac06d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb4045640 1 --2- 172.21.15.52:0/3617165576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac06d7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009870 tx=0x7f5c9c02a1c0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb4045640 1 -- 172.21.15.52:0/3617165576 shutdown_connections 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb4045640 1 --2- 172.21.15.52:0/3617165576 >> v2:172.21.15.119:3300/0 conn(0x7f5cac09c130 0x7f5cac0b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb4045640 1 --2- 172.21.15.52:0/3617165576 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac06d7a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb4045640 1 -- 172.21.15.52:0/3617165576 >> 172.21.15.52:0/3617165576 conn(0x7f5cac0b4a60 msgr2=0x7f5cac0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb4045640 1 -- 172.21.15.52:0/3617165576 shutdown_connections 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.791+0000 7f5cb4045640 1 -- 172.21.15.52:0/3617165576 wait complete. 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.792+0000 7f5cb4045640 1 Processor -- start 2024-05-11T01:13:25.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb4045640 1 -- start start 2024-05-11T01:13:25.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb4045640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cac09c130 0x7f5cac15e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb4045640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cac0b4320 0x7f5cac0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb4045640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac1f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb4045640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac0b5980 con 0x7f5cac09c130 2024-05-11T01:13:25.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb4045640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac0b5800 con 0x7f5cac0b4320 2024-05-11T01:13:25.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb4045640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5cac0d2380 con 0x7f5cac0d2b40 2024-05-11T01:13:25.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb1de1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cac09c130 0x7f5cac15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb1de1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cac09c130 0x7f5cac15e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50108/0 (socket says 172.21.15.52:50108) 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.793+0000 7f5cb1de1640 1 -- 172.21.15.52:0/2999127978 learned_addr learned my addr 172.21.15.52:0/2999127978 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb15e0640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cac0b4320 0x7f5cac0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb25e2640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac1f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb1de1640 1 -- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 msgr2=0x7f5cac1f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb1de1640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac1f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb1de1640 1 -- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cac0b4320 msgr2=0x7f5cac0c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb1de1640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cac0b4320 0x7f5cac0c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb1de1640 1 -- 172.21.15.52:0/2999127978 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac0c7980 con 0x7f5cac09c130 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb1de1640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cac09c130 0x7f5cac15e850 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800c490 tx=0x7f5ca800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5ca27fc640 1 -- 172.21.15.52:0/2999127978 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8018070 con 0x7f5cac09c130 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5ca27fc640 1 -- 172.21.15.52:0/2999127978 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca8007500 con 0x7f5cac09c130 2024-05-11T01:13:25.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac160fa0 con 0x7f5cac09c130 2024-05-11T01:13:25.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5ca27fc640 1 -- 172.21.15.52:0/2999127978 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8004a20 con 0x7f5cac09c130 2024-05-11T01:13:25.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.794+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac161840 con 0x7f5cac09c130 2024-05-11T01:13:25.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.795+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cac161170 con 0x7f5cac09c130 2024-05-11T01:13:25.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.796+0000 7f5ca27fc640 1 -- 172.21.15.52:0/2999127978 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f5ca8004bc0 con 0x7f5cac09c130 2024-05-11T01:13:25.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.796+0000 7f5ca27fc640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c900764d0 0x7f5c90078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:25.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.796+0000 7f5ca27fc640 1 -- 172.21.15.52:0/2999127978 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f5ca8096f80 con 0x7f5cac09c130 2024-05-11T01:13:25.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.797+0000 7f5cb15e0640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c900764d0 0x7f5c90078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:25.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.797+0000 7f5cb15e0640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c900764d0 0x7f5c90078990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c034000 tx=0x7f5c9c02e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:25.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:25.801+0000 7f5ca27fc640 1 -- 172.21.15.52:0/2999127978 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5ca8060440 con 0x7f5cac09c130 2024-05-11T01:13:26.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.016+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cac1613a0 con 0x7f5c900764d0 2024-05-11T01:13:26.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.024+0000 7f5ca27fc640 1 -- 172.21.15.52:0/2999127978 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f5c9c004340 con 0x7f5c900764d0 2024-05-11T01:13:26.023 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:26.023 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:01", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:26.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c900764d0 msgr2=0x7f5c90078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c900764d0 0x7f5c90078990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c034000 tx=0x7f5c9c02e000 comp rx=0 tx=0).stop 2024-05-11T01:13:26.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cac09c130 msgr2=0x7f5cac15e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cac09c130 0x7f5cac15e850 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800c490 tx=0x7f5ca800c950 comp rx=0 tx=0).stop 2024-05-11T01:13:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 shutdown_connections 2024-05-11T01:13:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cac0d2b40 0x7f5cac1f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cac0b4320 0x7f5cac0c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c900764d0 0x7f5c90078990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 --2- 172.21.15.52:0/2999127978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cac09c130 0x7f5cac15e850 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 >> 172.21.15.52:0/2999127978 conn(0x7f5cac0b4a60 msgr2=0x7f5cac163160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 shutdown_connections 2024-05-11T01:13:26.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.026+0000 7f5cb4045640 1 -- 172.21.15.52:0/2999127978 wait complete. 2024-05-11T01:13:26.036 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:01", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:26.036 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:02:01', 'first': '2024-05-11T01:03:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:26.036 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-05-11T01:13:26.036 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-05-11T01:13:26.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:26.036 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2024-05-11T01:13:26.057 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:26.057 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:26.057 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:26.057 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:26.058 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:26.058 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:26.058 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:26.058 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:26.058 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:26.058 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:26.058 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:26.058 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_00_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:26.058 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2024-05-11T01:13:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.207+0000 7fa81649b640 1 Processor -- start 2024-05-11T01:13:26.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.207+0000 7fa81649b640 1 -- start start 2024-05-11T01:13:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.208+0000 7fa81649b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa81006cb90 0x7fa8100b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.208+0000 7fa81649b640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa81006f700 con 0x7fa81015d0e0 2024-05-11T01:13:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.208+0000 7fa81649b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa81006f9e0 con 0x7fa810072330 2024-05-11T01:13:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.208+0000 7fa81649b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa81006ff80 con 0x7fa81006cb90 2024-05-11T01:13:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.208+0000 7fa80ffff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa81015d0e0 0x7fa8100ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55360/0 (socket says 172.21.15.52:55360) 2024-05-11T01:13:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.208+0000 7fa80ffff640 1 -- 172.21.15.52:0/2157999205 learned_addr learned my addr 172.21.15.52:0/2157999205 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.208+0000 7fa814a38640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa81006cb90 0x7fa8100b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.208+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2274659913 0 0) 0x7fa81006f700 con 0x7fa81015d0e0 2024-05-11T01:13:26.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.208+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7f8003520 con 0x7fa81015d0e0 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1610427527 0 0) 0x7fa7f8003520 con 0x7fa81015d0e0 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa81006cb90 msgr2=0x7fa8100b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa81006cb90 0x7fa8100b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 >> v1:172.21.15.119:6789/0 conn(0x7fa810072330 legacy=0x7fa81006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa810070260 con 0x7fa81015d0e0 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fa8000016f0 con 0x7fa81015d0e0 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 >> v1:172.21.15.52:6789/0 conn(0x7fa81015d0e0 legacy=0x7fa8100ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa81006cb90 0x7fa7f8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa7f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8004d40 0x7fa7f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa81006ff80 con 0x7fa7f8004000 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f800bb50 con 0x7fa7f8004d40 2024-05-11T01:13:26.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f800bd50 con 0x7fa81006cb90 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80f7fe640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa7f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80ffff640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa81006cb90 0x7fa7f8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa814a38640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8004d40 0x7fa7f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80f7fe640 1 -- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa81006cb90 msgr2=0x7fa7f8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80f7fe640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa81006cb90 0x7fa7f8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80f7fe640 1 -- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8004d40 msgr2=0x7fa7f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80f7fe640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8004d40 0x7fa7f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.209+0000 7fa80f7fe640 1 -- 172.21.15.52:0/2157999205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f8003520 con 0x7fa7f8004000 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.210+0000 7fa80f7fe640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa7f8004800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa804001790 tx=0x7fa7f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efc03ffeae5896f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.210+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8040013b0 con 0x7fa7f8004000 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.210+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8040052d0 con 0x7fa7f8004000 2024-05-11T01:13:26.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.210+0000 7fa80effd640 1 -- 172.21.15.52:0/2157999205 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80402aae0 con 0x7fa7f8004000 2024-05-11T01:13:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.211+0000 7fa81649b640 1 -- 172.21.15.52:0/2157999205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 msgr2=0x7fa7f8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.211+0000 7fa81649b640 1 --2- 172.21.15.52:0/2157999205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa7f8004800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa804001790 tx=0x7fa7f8005830 comp rx=0 tx=0).stop 2024-05-11T01:13:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.211+0000 7fa814a38640 1 -- 172.21.15.52:0/2157999205 reap_dead start 2024-05-11T01:13:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.211+0000 7fa81649b640 1 -- 172.21.15.52:0/2157999205 shutdown_connections 2024-05-11T01:13:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.211+0000 7fa81649b640 1 -- 172.21.15.52:0/2157999205 >> 172.21.15.52:0/2157999205 conn(0x7fa8100cd830 msgr2=0x7fa8100cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.211+0000 7fa81649b640 1 -- 172.21.15.52:0/2157999205 shutdown_connections 2024-05-11T01:13:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.211+0000 7fa81649b640 1 -- 172.21.15.52:0/2157999205 wait complete. 2024-05-11T01:13:26.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.212+0000 7fa81649b640 1 Processor -- start 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.212+0000 7fa81649b640 1 -- start start 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa81649b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa8100cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa81649b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8100cd100 0x7fa8100ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa81649b640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8100cc0b0 0x7fa8100c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa81649b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8100b0380 con 0x7fa7f8004000 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa81649b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa8100b0200 con 0x7fa8100cd100 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa81649b640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa8100b0500 con 0x7fa8100cc0b0 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa8100cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa8100cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50122/0 (socket says 172.21.15.52:50122) 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80ffff640 1 -- 172.21.15.52:0/2318079579 learned_addr learned my addr 172.21.15.52:0/2318079579 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80f7fe640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8100cd100 0x7fa8100ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa814a38640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8100cc0b0 0x7fa8100c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80ffff640 1 -- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8100cc0b0 msgr2=0x7fa8100c3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80ffff640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8100cc0b0 0x7fa8100c3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80ffff640 1 -- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8100cd100 msgr2=0x7fa8100ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80ffff640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8100cd100 0x7fa8100ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80ffff640 1 -- 172.21.15.52:0/2318079579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8101fd080 con 0x7fa7f8004000 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.213+0000 7fa80ffff640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa8100cb400 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa800007310 tx=0x7fa8000077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.214+0000 7fa80cff9640 1 -- 172.21.15.52:0/2318079579 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80001a070 con 0x7fa7f8004000 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.214+0000 7fa80cff9640 1 -- 172.21.15.52:0/2318079579 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80000f940 con 0x7fa7f8004000 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.214+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8101fd360 con 0x7fa7f8004000 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.214+0000 7fa80cff9640 1 -- 172.21.15.52:0/2318079579 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80000fd50 con 0x7fa7f8004000 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.214+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8101fd920 con 0x7fa7f8004000 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.215+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8100695d0 con 0x7fa7f8004000 2024-05-11T01:13:26.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.215+0000 7fa80cff9640 1 -- 172.21.15.52:0/2318079579 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fa80001d070 con 0x7fa7f8004000 2024-05-11T01:13:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.216+0000 7fa80cff9640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa7fc0764d0 0x7fa7fc078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.216+0000 7fa80cff9640 1 -- 172.21.15.52:0/2318079579 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7fa800097480 con 0x7fa7f8004000 2024-05-11T01:13:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.216+0000 7fa80f7fe640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa7fc0764d0 0x7fa7fc078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.217+0000 7fa80f7fe640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa7fc0764d0 0x7fa7fc078990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa804001790 tx=0x7fa80402e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:26.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.220+0000 7fa80cff9640 1 -- 172.21.15.52:0/2318079579 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa800060a60 con 0x7fa7f8004000 2024-05-11T01:13:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.434+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8100d02e0 con 0x7fa7fc0764d0 2024-05-11T01:13:26.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.442+0000 7fa80cff9640 1 -- 172.21.15.52:0/2318079579 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7fa8100d02e0 con 0x7fa7fc0764d0 2024-05-11T01:13:26.442 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:26.443 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:01", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa7fc0764d0 msgr2=0x7fa7fc078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa7fc0764d0 0x7fa7fc078990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa804001790 tx=0x7fa80402e000 comp rx=0 tx=0).stop 2024-05-11T01:13:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 msgr2=0x7fa8100cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa8100cb400 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa800007310 tx=0x7fa8000077d0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 shutdown_connections 2024-05-11T01:13:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa8100cc0b0 0x7fa8100c3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8100cd100 0x7fa8100ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa7fc0764d0 0x7fa7fc078990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 --2- 172.21.15.52:0/2318079579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7f8004000 0x7fa8100cb400 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.444+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 >> 172.21.15.52:0/2318079579 conn(0x7fa8100cd830 msgr2=0x7fa8100ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:26.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.445+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 shutdown_connections 2024-05-11T01:13:26.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.445+0000 7fa81649b640 1 -- 172.21.15.52:0/2318079579 wait complete. 2024-05-11T01:13:26.454 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:01", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:26.454 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:02:01', 'first': '2024-05-11T01:03:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:26.454 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-05-11T01:13:26.455 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-05-11T01:13:26.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:26.455 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:26.476 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:26.477 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_00_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:26.477 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2024-05-11T01:13:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.622+0000 7fc5e27c1640 1 Processor -- start 2024-05-11T01:13:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.623+0000 7fc5e27c1640 1 -- start start 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.623+0000 7fc5e27c1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 0x7fc5dc0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.623+0000 7fc5e27c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5dc0d4e40 0x7fc5dc0d5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.623+0000 7fc5e27c1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc5dc0d2d60 0x7fc5dc0d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.623+0000 7fc5e27c1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc5dc0d3a90 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.623+0000 7fc5e27c1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fc5dc0d3d70 con 0x7fc5dc0d4e40 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.623+0000 7fc5e27c1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc0d4050 con 0x7fc5dc15f2f0 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5e0d5e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc5dc0d2d60 0x7fc5dc0d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5e0d5e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc5dc0d2d60 0x7fc5dc0d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50124/0 (socket says 172.21.15.52:50124) 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5e0d5e640 1 -- 172.21.15.52:0/1955558895 learned_addr learned my addr 172.21.15.52:0/1955558895 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5dbfff640 1 --2- 172.21.15.52:0/1955558895 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 0x7fc5dc0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5db7fe640 1 --2- 172.21.15.52:0/1955558895 >> v2:172.21.15.119:3300/0 conn(0x7fc5dc0d4e40 0x7fc5dc0d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5e0d5e640 1 -- 172.21.15.52:0/1955558895 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 msgr2=0x7fc5dc0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5e0d5e640 1 --2- 172.21.15.52:0/1955558895 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 0x7fc5dc0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5e0d5e640 1 -- 172.21.15.52:0/1955558895 >> v2:172.21.15.119:3300/0 conn(0x7fc5dc0d4e40 msgr2=0x7fc5dc0d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5e0d5e640 1 --2- 172.21.15.52:0/1955558895 >> v2:172.21.15.119:3300/0 conn(0x7fc5dc0d4e40 0x7fc5dc0d5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5e0d5e640 1 -- 172.21.15.52:0/1955558895 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc0b4e60 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.624+0000 7fc5db7fe640 1 --2- 172.21.15.52:0/1955558895 >> v2:172.21.15.119:3300/0 conn(0x7fc5dc0d4e40 0x7fc5dc0d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e0d5e640 1 --2- 172.21.15.52:0/1955558895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 0x7fc5dc0d31d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0d3f80 tx=0x7fc5d402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f416ba2e33162d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5daffd640 1 -- 172.21.15.52:0/1955558895 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4036070 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5daffd640 1 -- 172.21.15.52:0/1955558895 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d4031600 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5daffd640 1 -- 172.21.15.52:0/1955558895 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d40319d0 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1955558895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 msgr2=0x7fc5dc0d31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1955558895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 0x7fc5dc0d31d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0d3f80 tx=0x7fc5d402a680 comp rx=0 tx=0).stop 2024-05-11T01:13:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1955558895 shutdown_connections 2024-05-11T01:13:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1955558895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 0x7fc5dc0d31d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1955558895 >> v2:172.21.15.119:3300/0 conn(0x7fc5dc0d4e40 0x7fc5dc0d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1955558895 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 0x7fc5dc0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1955558895 >> 172.21.15.52:0/1955558895 conn(0x7fc5dc08ebb0 msgr2=0x7fc5dc09e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1955558895 shutdown_connections 2024-05-11T01:13:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.625+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1955558895 wait complete. 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.627+0000 7fc5e27c1640 1 Processor -- start 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.627+0000 7fc5e27c1640 1 -- start start 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.627+0000 7fc5e27c1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 0x7fc5dc0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.627+0000 7fc5e27c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5dc0d4e40 0x7fc5dc15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5e27c1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 0x7fc5dc15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5e27c1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc0d3710 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5e27c1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc0d57c0 con 0x7fc5dc0d4e40 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5e27c1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc0d01e0 con 0x7fc5dc15f2f0 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 0x7fc5dc0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 0x7fc5dc0cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50126/0 (socket says 172.21.15.52:50126) 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5dbfff640 1 -- 172.21.15.52:0/1371384998 learned_addr learned my addr 172.21.15.52:0/1371384998 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5db7fe640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5dc0d4e40 0x7fc5dc15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5e0d5e640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 0x7fc5dc15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5dbfff640 1 -- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 msgr2=0x7fc5dc15c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5dbfff640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 0x7fc5dc15c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5dbfff640 1 -- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5dc0d4e40 msgr2=0x7fc5dc15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5dbfff640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5dc0d4e40 0x7fc5dc15f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5dbfff640 1 -- 172.21.15.52:0/1371384998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc0a0670 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5db7fe640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5dc0d4e40 0x7fc5dc15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5e0d5e640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 0x7fc5dc15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5dbfff640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 0x7fc5dc0cb760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00e9b0 tx=0x7fc5cc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5d8ff9640 1 -- 172.21.15.52:0/1371384998 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc017070 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5d8ff9640 1 -- 172.21.15.52:0/1371384998 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc00c9a0 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc0a0950 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5d8ff9640 1 -- 172.21.15.52:0/1371384998 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc00cd90 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.628+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc0a0f10 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.630+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc0695d0 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.630+0000 7fc5d8ff9640 1 -- 172.21.15.52:0/1371384998 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fc5cc01a070 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.631+0000 7fc5d8ff9640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5c80764d0 0x7fc5c8078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.631+0000 7fc5d8ff9640 1 -- 172.21.15.52:0/1371384998 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7fc5cc097e40 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.631+0000 7fc5db7fe640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5c80764d0 0x7fc5c8078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.631+0000 7fc5db7fe640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5c80764d0 0x7fc5c8078990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0d2bd0 tx=0x7fc5d0008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:26.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.635+0000 7fc5d8ff9640 1 -- 172.21.15.52:0/1371384998 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc5cc061420 con 0x7fc5dc0d2d60 2024-05-11T01:13:26.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.847+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5dc0c8a70 con 0x7fc5c80764d0 2024-05-11T01:13:26.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.855+0000 7fc5d8ff9640 1 -- 172.21.15.52:0/1371384998 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7fc5dc0c8a70 con 0x7fc5c80764d0 2024-05-11T01:13:26.855 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:26.855 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:02", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:26.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5c80764d0 msgr2=0x7fc5c8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5c80764d0 0x7fc5c8078990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0d2bd0 tx=0x7fc5d0008040 comp rx=0 tx=0).stop 2024-05-11T01:13:26.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 msgr2=0x7fc5dc0cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:26.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 0x7fc5dc0cb760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00e9b0 tx=0x7fc5cc00ee70 comp rx=0 tx=0).stop 2024-05-11T01:13:26.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 shutdown_connections 2024-05-11T01:13:26.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5dc15f2f0 0x7fc5dc15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5dc0d4e40 0x7fc5dc15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5c80764d0 0x7fc5c8078990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 --2- 172.21.15.52:0/1371384998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5dc0d2d60 0x7fc5dc0cb760 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:26.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.857+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 >> 172.21.15.52:0/1371384998 conn(0x7fc5dc08ebb0 msgr2=0x7fc5dc06dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:26.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.858+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 shutdown_connections 2024-05-11T01:13:26.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:26.858+0000 7fc5e27c1640 1 -- 172.21.15.52:0/1371384998 wait complete. 2024-05-11T01:13:26.867 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:02", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:26.867 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:02:02', 'first': '2024-05-11T01:03:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:26.867 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-05-11T01:13:26.868 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-05-11T01:13:26.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:26.868 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2024-05-11T01:13:26.888 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:26.889 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:26.889 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_00_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:26.890 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2024-05-11T01:13:27.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.035+0000 7fc553859640 1 Processor -- start 2024-05-11T01:13:27.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.035+0000 7fc553859640 1 -- start start 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc553859640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 0x7fc54c06d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc553859640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc54c09c130 0x7fc54c0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc553859640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc54c06e140 con 0x7fc54c0d2b40 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc553859640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc54c06dd80 con 0x7fc54c09c130 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc553859640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c06e420 con 0x7fc54c0b4320 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc551df6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc54c09c130 0x7fc54c0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc550df4640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc54c0d2b40 0x7fc54c09ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55362/0 (socket says 172.21.15.52:55362) 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc551df6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc54c09c130 0x7fc54c0b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50142/0 (socket says 172.21.15.52:50142) 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc550df4640 1 -- 172.21.15.52:0/1096712069 learned_addr learned my addr 172.21.15.52:0/1096712069 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc5515f5640 1 --2- 172.21.15.52:0/1096712069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 0x7fc54c06d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.036+0000 7fc551df6640 1 -- 172.21.15.52:0/1096712069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 msgr2=0x7fc54c06d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.037+0000 7fc551df6640 1 --2- 172.21.15.52:0/1096712069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 0x7fc54c06d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.037+0000 7fc551df6640 1 -- 172.21.15.52:0/1096712069 >> v1:172.21.15.52:6789/0 conn(0x7fc54c0d2b40 legacy=0x7fc54c09ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.037+0000 7fc551df6640 1 -- 172.21.15.52:0/1096712069 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c0d1e60 con 0x7fc54c09c130 2024-05-11T01:13:27.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.037+0000 7fc543fff640 1 -- 172.21.15.52:0/1096712069 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772094566 0 0) 0x7fc54c06e140 con 0x7fc54c0d2b40 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.037+0000 7fc551df6640 1 --2- 172.21.15.52:0/1096712069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c09c130 0x7fc54c0b28d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc544009d90 tx=0x7fc54402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2562175c004f7e3d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.037+0000 7fc543fff640 1 -- 172.21.15.52:0/1096712069 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544031600 con 0x7fc54c09c130 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.037+0000 7fc543fff640 1 -- 172.21.15.52:0/1096712069 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5440317a0 con 0x7fc54c09c130 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.037+0000 7fc543fff640 1 -- 172.21.15.52:0/1096712069 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544031b70 con 0x7fc54c09c130 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.038+0000 7fc553859640 1 -- 172.21.15.52:0/1096712069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c09c130 msgr2=0x7fc54c0b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.038+0000 7fc553859640 1 --2- 172.21.15.52:0/1096712069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c09c130 0x7fc54c0b28d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc544009d90 tx=0x7fc54402a5d0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.038+0000 7fc553859640 1 -- 172.21.15.52:0/1096712069 shutdown_connections 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.038+0000 7fc553859640 1 --2- 172.21.15.52:0/1096712069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c09c130 0x7fc54c0b28d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.038+0000 7fc553859640 1 --2- 172.21.15.52:0/1096712069 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 0x7fc54c06d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.038+0000 7fc553859640 1 -- 172.21.15.52:0/1096712069 >> 172.21.15.52:0/1096712069 conn(0x7fc54c0b4a60 msgr2=0x7fc54c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:27.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.038+0000 7fc553859640 1 -- 172.21.15.52:0/1096712069 shutdown_connections 2024-05-11T01:13:27.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.038+0000 7fc553859640 1 -- 172.21.15.52:0/1096712069 wait complete. 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.039+0000 7fc553859640 1 Processor -- start 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.039+0000 7fc553859640 1 -- start start 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc553859640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54c09c130 0x7fc54c1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc553859640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 0x7fc54c1f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc553859640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c0d2b40 0x7fc54c0b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc553859640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0b5980 con 0x7fc54c0d2b40 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc553859640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0b5800 con 0x7fc54c09c130 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc553859640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c0d2380 con 0x7fc54c0b4320 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc551df6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c0d2b40 0x7fc54c0b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc551df6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c0d2b40 0x7fc54c0b70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50154/0 (socket says 172.21.15.52:50154) 2024-05-11T01:13:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc551df6640 1 -- 172.21.15.52:0/1065841642 learned_addr learned my addr 172.21.15.52:0/1065841642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc550df4640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 0x7fc54c1f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.040+0000 7fc5515f5640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54c09c130 0x7fc54c1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc551df6640 1 -- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 msgr2=0x7fc54c1f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc551df6640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 0x7fc54c1f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc551df6640 1 -- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54c09c130 msgr2=0x7fc54c1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc551df6640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54c09c130 0x7fc54c1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc551df6640 1 -- 172.21.15.52:0/1065841642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c160980 con 0x7fc54c0d2b40 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc551df6640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c0d2b40 0x7fc54c0b70f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc544009eb0 tx=0x7fc544004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc541ffb640 1 -- 172.21.15.52:0/1065841642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544040070 con 0x7fc54c0d2b40 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc541ffb640 1 -- 172.21.15.52:0/1065841642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc544030b60 con 0x7fc54c0d2b40 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c160c60 con 0x7fc54c0d2b40 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc541ffb640 1 -- 172.21.15.52:0/1065841642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544009450 con 0x7fc54c0d2b40 2024-05-11T01:13:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.041+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c161220 con 0x7fc54c0d2b40 2024-05-11T01:13:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.042+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c0695d0 con 0x7fc54c0d2b40 2024-05-11T01:13:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.043+0000 7fc541ffb640 1 -- 172.21.15.52:0/1065841642 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fc544039040 con 0x7fc54c0d2b40 2024-05-11T01:13:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.043+0000 7fc541ffb640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5300764d0 0x7fc530078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.044+0000 7fc541ffb640 1 -- 172.21.15.52:0/1065841642 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7fc5440b9210 con 0x7fc54c0d2b40 2024-05-11T01:13:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.044+0000 7fc5515f5640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5300764d0 0x7fc530078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.044+0000 7fc5515f5640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5300764d0 0x7fc530078990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc548006fd0 tx=0x7fc548008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.048+0000 7fc541ffb640 1 -- 172.21.15.52:0/1065841642 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc544081ff0 con 0x7fc54c0d2b40 2024-05-11T01:13:27.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.261+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc54c0b2500 con 0x7fc5300764d0 2024-05-11T01:13:27.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.269+0000 7fc541ffb640 1 -- 172.21.15.52:0/1065841642 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7fc54c0b2500 con 0x7fc5300764d0 2024-05-11T01:13:27.268 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:27.268 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:02", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5300764d0 msgr2=0x7fc530078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5300764d0 0x7fc530078990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc548006fd0 tx=0x7fc548008040 comp rx=0 tx=0).stop 2024-05-11T01:13:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c0d2b40 msgr2=0x7fc54c0b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c0d2b40 0x7fc54c0b70f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc544009eb0 tx=0x7fc544004a40 comp rx=0 tx=0).stop 2024-05-11T01:13:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 shutdown_connections 2024-05-11T01:13:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fc5300764d0 0x7fc530078990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc54c0d2b40 0x7fc54c0b70f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc54c0b4320 0x7fc54c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 --2- 172.21.15.52:0/1065841642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc54c09c130 0x7fc54c1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 >> 172.21.15.52:0/1065841642 conn(0x7fc54c0b4a60 msgr2=0x7fc54c070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:27.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 shutdown_connections 2024-05-11T01:13:27.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.272+0000 7fc553859640 1 -- 172.21.15.52:0/1065841642 wait complete. 2024-05-11T01:13:27.281 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:02", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:27.282 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:02:02', 'first': '2024-05-11T01:03:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:27.282 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-05-11T01:13:27.282 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-05-11T01:13:27.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:27.282 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2024-05-11T01:13:27.303 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:27.303 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:27.303 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:27.303 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:27.303 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:27.303 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:27.303 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:27.304 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:27.304 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:27.304 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:27.304 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:27.304 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-05-11-01_03_00_UTC', 'scheduled-2024-05-11-01_04_00_UTC', 'scheduled-2024-05-11-01_05_00_UTC', 'scheduled-2024-05-11-01_06_00_UTC', 'scheduled-2024-05-11-01_07_00_UTC', 'scheduled-2024-05-11-01_08_00_UTC', 'scheduled-2024-05-11-01_09_00_UTC', 'scheduled-2024-05-11-01_10_00_UTC', 'scheduled-2024-05-11-01_11_00_UTC', 'scheduled-2024-05-11-01_12_00_UTC', 'scheduled-2024-05-11-01_13_00_UTC'] 2024-05-11T01:13:27.304 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2024-05-11T01:13:27.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.451+0000 7ffbcee72640 1 Processor -- start 2024-05-11T01:13:27.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.451+0000 7ffbcee72640 1 -- start start 2024-05-11T01:13:27.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.451+0000 7ffbcee72640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc815f2f0 0x7ffbc80cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.451+0000 7ffbcee72640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffbc80d4e40 0x7ffbc80d5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.451+0000 7ffbcee72640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbc80d2d60 0x7ffbc80d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.451+0000 7ffbcee72640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ffbc80d3a90 con 0x7ffbc80d4e40 2024-05-11T01:13:27.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.451+0000 7ffbcee72640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffbc80d3d70 con 0x7ffbc80d2d60 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.451+0000 7ffbcee72640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc80d4050 con 0x7ffbc815f2f0 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbc7fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffbc80d4e40 0x7ffbc80d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbc7fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ffbc80d4e40 0x7ffbc80d5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50166/0 (socket says 172.21.15.52:50166) 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbc7fff640 1 -- 172.21.15.52:0/1635392727 learned_addr learned my addr 172.21.15.52:0/1635392727 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbcd40f640 1 --2- 172.21.15.52:0/1635392727 >> v2:172.21.15.119:3300/0 conn(0x7ffbc80d2d60 0x7ffbc80d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbccc0e640 1 --2- 172.21.15.52:0/1635392727 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc815f2f0 0x7ffbc80cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbc7fff640 1 -- 172.21.15.52:0/1635392727 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc815f2f0 msgr2=0x7ffbc80cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbc7fff640 1 --2- 172.21.15.52:0/1635392727 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc815f2f0 0x7ffbc80cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbc7fff640 1 -- 172.21.15.52:0/1635392727 >> v2:172.21.15.119:3300/0 conn(0x7ffbc80d2d60 msgr2=0x7ffbc80d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbc7fff640 1 --2- 172.21.15.52:0/1635392727 >> v2:172.21.15.119:3300/0 conn(0x7ffbc80d2d60 0x7ffbc80d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbc7fff640 1 -- 172.21.15.52:0/1635392727 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc80b4e60 con 0x7ffbc80d4e40 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbccc0e640 1 --2- 172.21.15.52:0/1635392727 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc815f2f0 0x7ffbc80cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:27.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.452+0000 7ffbcd40f640 1 --2- 172.21.15.52:0/1635392727 >> v2:172.21.15.119:3300/0 conn(0x7ffbc80d2d60 0x7ffbc80d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:27.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbc7fff640 1 --2- 172.21.15.52:0/1635392727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc80d4e40 0x7ffbc80d5280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80d3f80 tx=0x7ffbb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c620a481f9bda4a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbc77fe640 1 -- 172.21.15.52:0/1635392727 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4037070 con 0x7ffbc80d4e40 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbc77fe640 1 -- 172.21.15.52:0/1635392727 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb4032440 con 0x7ffbc80d4e40 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbc77fe640 1 -- 172.21.15.52:0/1635392727 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4032810 con 0x7ffbc80d4e40 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbcee72640 1 -- 172.21.15.52:0/1635392727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc80d4e40 msgr2=0x7ffbc80d5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbcee72640 1 --2- 172.21.15.52:0/1635392727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc80d4e40 0x7ffbc80d5280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80d3f80 tx=0x7ffbb402a1e0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbcee72640 1 -- 172.21.15.52:0/1635392727 shutdown_connections 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbcee72640 1 --2- 172.21.15.52:0/1635392727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc80d4e40 0x7ffbc80d5280 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbcee72640 1 --2- 172.21.15.52:0/1635392727 >> v2:172.21.15.119:3300/0 conn(0x7ffbc80d2d60 0x7ffbc80d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbcee72640 1 --2- 172.21.15.52:0/1635392727 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc815f2f0 0x7ffbc80cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbcee72640 1 -- 172.21.15.52:0/1635392727 >> 172.21.15.52:0/1635392727 conn(0x7ffbc808ebb0 msgr2=0x7ffbc809e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbcee72640 1 -- 172.21.15.52:0/1635392727 shutdown_connections 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.453+0000 7ffbcee72640 1 -- 172.21.15.52:0/1635392727 wait complete. 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.455+0000 7ffbcee72640 1 Processor -- start 2024-05-11T01:13:27.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.455+0000 7ffbcee72640 1 -- start start 2024-05-11T01:13:27.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.455+0000 7ffbcee72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc80d2d60 0x7ffbc80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcee72640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc80d4e40 0x7ffbc815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcee72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc815f2f0 0x7ffbc80c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcee72640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc80d3710 con 0x7ffbc815f2f0 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcee72640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc80d57c0 con 0x7ffbc80d2d60 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcee72640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc80d01e0 con 0x7ffbc80d4e40 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcd40f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc815f2f0 0x7ffbc80c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcd40f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc815f2f0 0x7ffbc80c96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50182/0 (socket says 172.21.15.52:50182) 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcd40f640 1 -- 172.21.15.52:0/4008529667 learned_addr learned my addr 172.21.15.52:0/4008529667 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbccc0e640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc80d2d60 0x7ffbc80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbc7fff640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc80d4e40 0x7ffbc815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcd40f640 1 -- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc80d4e40 msgr2=0x7ffbc815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcd40f640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc80d4e40 0x7ffbc815c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcd40f640 1 -- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc80d2d60 msgr2=0x7ffbc80c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcd40f640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc80d2d60 0x7ffbc80c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcd40f640 1 -- 172.21.15.52:0/4008529667 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc815c790 con 0x7ffbc815f2f0 2024-05-11T01:13:27.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcd40f640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc815f2f0 0x7ffbc80c96d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000cd50 tx=0x7ffbc000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:27.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbc57fa640 1 -- 172.21.15.52:0/4008529667 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc000f680 con 0x7ffbc815f2f0 2024-05-11T01:13:27.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbc57fa640 1 -- 172.21.15.52:0/4008529667 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc000f840 con 0x7ffbc815f2f0 2024-05-11T01:13:27.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbc57fa640 1 -- 172.21.15.52:0/4008529667 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc000fc30 con 0x7ffbc815f2f0 2024-05-11T01:13:27.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc80ce280 con 0x7ffbc815f2f0 2024-05-11T01:13:27.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.456+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc80ce840 con 0x7ffbc815f2f0 2024-05-11T01:13:27.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.457+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc80695d0 con 0x7ffbc815f2f0 2024-05-11T01:13:27.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.458+0000 7ffbc57fa640 1 -- 172.21.15.52:0/4008529667 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7ffbc00213f0 con 0x7ffbc815f2f0 2024-05-11T01:13:27.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.459+0000 7ffbc57fa640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffbac0764d0 0x7ffbac078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.459+0000 7ffbc57fa640 1 -- 172.21.15.52:0/4008529667 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7ffbc00988f0 con 0x7ffbc815f2f0 2024-05-11T01:13:27.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.459+0000 7ffbccc0e640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffbac0764d0 0x7ffbac078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.459+0000 7ffbccc0e640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffbac0764d0 0x7ffbac078990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80b86b0 tx=0x7ffbbc008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:27.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.463+0000 7ffbc57fa640 1 -- 172.21.15.52:0/4008529667 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffbc815c790 con 0x7ffbc815f2f0 2024-05-11T01:13:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.680+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbc80d6080 con 0x7ffbac0764d0 2024-05-11T01:13:27.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.688+0000 7ffbc57fa640 1 -- 172.21.15.52:0/4008529667 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7ffbc80d6080 con 0x7ffbac0764d0 2024-05-11T01:13:27.687 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:27.687 INFO:teuthology.orchestra.run.smithi052.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:03", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:27.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffbac0764d0 msgr2=0x7ffbac078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffbac0764d0 0x7ffbac078990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80b86b0 tx=0x7ffbbc008040 comp rx=0 tx=0).stop 2024-05-11T01:13:27.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc815f2f0 msgr2=0x7ffbc80c96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc815f2f0 0x7ffbc80c96d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000cd50 tx=0x7ffbc000a650 comp rx=0 tx=0).stop 2024-05-11T01:13:27.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 shutdown_connections 2024-05-11T01:13:27.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ffbac0764d0 0x7ffbac078990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc815f2f0 0x7ffbc80c96d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffbc80d4e40 0x7ffbc815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 --2- 172.21.15.52:0/4008529667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc80d2d60 0x7ffbc80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 >> 172.21.15.52:0/4008529667 conn(0x7ffbc808ebb0 msgr2=0x7ffbc806dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:27.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 shutdown_connections 2024-05-11T01:13:27.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.691+0000 7ffbcee72640 1 -- 172.21.15.52:0/4008529667 wait complete. 2024-05-11T01:13:27.701 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2024-05-11T00:00:00", "created": "2024-05-11T01:02:03", "first": "2024-05-11T01:03:00", "last": "2024-05-11T01:13:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-05-11T01:13:27.701 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1m', 'retention': {}, 'start': '2024-05-11T00:00:00', 'created': '2024-05-11T01:02:03', 'first': '2024-05-11T01:03:00', 'last': '2024-05-11T01:13:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-05-11T01:13:27.701 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-05-11T01:13:27.701 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-05-11T01:13:27.701 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1m 2024-05-11T01:13:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.808+0000 7f2130ecc640 1 Processor -- start 2024-05-11T01:13:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.808+0000 7f2130ecc640 1 -- start start 2024-05-11T01:13:27.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f2130ecc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f212c0d57d0 0x7f212c0d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f2130ecc640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f212c15f8b0 con 0x7f212c15f4d0 2024-05-11T01:13:27.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f2130ecc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f212c0b4e40 con 0x7f212c0b5ee0 2024-05-11T01:13:27.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f2130ecc640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f212c0b5120 con 0x7f212c0d57d0 2024-05-11T01:13:27.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f2129d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f212c0d57d0 0x7f212c0d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f212a59c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f212c15f4d0 0x7f212c0d50c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55372/0 (socket says 172.21.15.52:55372) 2024-05-11T01:13:27.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f212a59c640 1 -- 172.21.15.52:0/3517718277 learned_addr learned my addr 172.21.15.52:0/3517718277 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:27.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f2129d9b640 1 -- 172.21.15.52:0/3517718277 >> v1:172.21.15.119:6789/0 conn(0x7f212c0b5ee0 legacy=0x7f212c0d3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f2129d9b640 1 -- 172.21.15.52:0/3517718277 >> v1:172.21.15.52:6789/0 conn(0x7f212c15f4d0 legacy=0x7f212c0d50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f2129d9b640 1 -- 172.21.15.52:0/3517718277 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c0b5400 con 0x7f212c0d57d0 2024-05-11T01:13:27.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.809+0000 7f212959a640 1 -- 172.21.15.52:0/3517718277 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3156194805 0 0) 0x7f212c15f8b0 con 0x7f212c15f4d0 2024-05-11T01:13:27.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.810+0000 7f2129d9b640 1 --2- 172.21.15.52:0/3517718277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 0x7f212c0d3710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f21200097a0 tx=0x7f212002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99109b8dbe690999 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:27.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.810+0000 7f212959a640 1 -- 172.21.15.52:0/3517718277 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120037070 con 0x7f212c0d57d0 2024-05-11T01:13:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.810+0000 7f212959a640 1 -- 172.21.15.52:0/3517718277 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2120032440 con 0x7f212c0d57d0 2024-05-11T01:13:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.810+0000 7f212959a640 1 -- 172.21.15.52:0/3517718277 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120032810 con 0x7f212c0d57d0 2024-05-11T01:13:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.811+0000 7f2130ecc640 1 -- 172.21.15.52:0/3517718277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 msgr2=0x7f212c0d3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.811+0000 7f2130ecc640 1 --2- 172.21.15.52:0/3517718277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 0x7f212c0d3710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f21200097a0 tx=0x7f212002a230 comp rx=0 tx=0).stop 2024-05-11T01:13:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.811+0000 7f2130ecc640 1 -- 172.21.15.52:0/3517718277 shutdown_connections 2024-05-11T01:13:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.811+0000 7f2130ecc640 1 --2- 172.21.15.52:0/3517718277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 0x7f212c0d3710 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.811+0000 7f2130ecc640 1 -- 172.21.15.52:0/3517718277 >> 172.21.15.52:0/3517718277 conn(0x7f212c08ebc0 msgr2=0x7f212c09e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.811+0000 7f2130ecc640 1 -- 172.21.15.52:0/3517718277 shutdown_connections 2024-05-11T01:13:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.811+0000 7f2130ecc640 1 -- 172.21.15.52:0/3517718277 wait complete. 2024-05-11T01:13:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.812+0000 7f2130ecc640 1 Processor -- start 2024-05-11T01:13:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.812+0000 7f2130ecc640 1 -- start start 2024-05-11T01:13:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2130ecc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f212c0b5ee0 0x7f212c14d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2130ecc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 0x7f212c155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2130ecc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f212c15f4d0 0x7f212c160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2130ecc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f212c169fe0 con 0x7f212c0d57d0 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2130ecc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f212c0b59c0 con 0x7f212c15f4d0 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2130ecc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f212c16a160 con 0x7f212c0b5ee0 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2129d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 0x7f212c155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2129d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 0x7f212c155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50200/0 (socket says 172.21.15.52:50200) 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2129d9b640 1 -- 172.21.15.52:0/83965689 learned_addr learned my addr 172.21.15.52:0/83965689 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f212a59c640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f212c0b5ee0 0x7f212c14d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f212ad9d640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f212c15f4d0 0x7f212c160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2129d9b640 1 -- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f212c0b5ee0 msgr2=0x7f212c14d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2129d9b640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f212c0b5ee0 0x7f212c14d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2129d9b640 1 -- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f212c15f4d0 msgr2=0x7f212c160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.813+0000 7f2129d9b640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f212c15f4d0 0x7f212c160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:27.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.814+0000 7f2129d9b640 1 -- 172.21.15.52:0/83965689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c161430 con 0x7f212c0d57d0 2024-05-11T01:13:27.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.814+0000 7f212ad9d640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f212c15f4d0 0x7f212c160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:27.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.814+0000 7f2129d9b640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 0x7f212c155530 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f21200097a0 tx=0x7f21200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:27.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.814+0000 7f211affd640 1 -- 172.21.15.52:0/83965689 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120037050 con 0x7f212c0d57d0 2024-05-11T01:13:27.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.814+0000 7f211affd640 1 -- 172.21.15.52:0/83965689 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2120038040 con 0x7f212c0d57d0 2024-05-11T01:13:27.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.814+0000 7f211affd640 1 -- 172.21.15.52:0/83965689 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120004490 con 0x7f212c0d57d0 2024-05-11T01:13:27.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.814+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212c161700 con 0x7f212c0d57d0 2024-05-11T01:13:27.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.814+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212c161cc0 con 0x7f212c0d57d0 2024-05-11T01:13:27.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.815+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212c0694f0 con 0x7f212c0d57d0 2024-05-11T01:13:27.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.815+0000 7f211affd640 1 -- 172.21.15.52:0/83965689 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f2120030900 con 0x7f212c0d57d0 2024-05-11T01:13:27.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.816+0000 7f211affd640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f210c0764d0 0x7f210c078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:27.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.816+0000 7f211affd640 1 -- 172.21.15.52:0/83965689 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f21200b8580 con 0x7f212c0d57d0 2024-05-11T01:13:27.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.816+0000 7f212a59c640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f210c0764d0 0x7f210c078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:27.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.817+0000 7f212a59c640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f210c0764d0 0x7f210c078990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f211c000f60 tx=0x7f211c007e40 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:27.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:27.821+0000 7f211affd640 1 -- 172.21.15.52:0/83965689 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2120081b50 con 0x7f212c0d57d0 2024-05-11T01:13:28.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.032+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f212c0d0b80 con 0x7f210c0764d0 2024-05-11T01:13:28.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.118+0000 7f211affd640 1 -- 172.21.15.52:0/83965689 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f212c0d0b80 con 0x7f210c0764d0 2024-05-11T01:13:28.161 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir0 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.120+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f210c0764d0 msgr2=0x7f210c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.120+0000 7f2130ecc640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f210c0764d0 0x7f210c078990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f211c000f60 tx=0x7f211c007e40 comp rx=0 tx=0).stop 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.120+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 msgr2=0x7f212c155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.120+0000 7f2130ecc640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 0x7f212c155530 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f21200097a0 tx=0x7f21200046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.120+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 shutdown_connections 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.120+0000 7f2130ecc640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f212c15f4d0 0x7f212c160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.120+0000 7f2130ecc640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f210c0764d0 0x7f210c078990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.120+0000 7f2130ecc640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f212c0d57d0 0x7f212c155530 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.120+0000 7f2130ecc640 1 --2- 172.21.15.52:0/83965689 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f212c0b5ee0 0x7f212c14d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.121+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 >> 172.21.15.52:0/83965689 conn(0x7f212c08ebc0 msgr2=0x7f212c06e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.121+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 shutdown_connections 2024-05-11T01:13:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:28.121+0000 7f2130ecc640 1 -- 172.21.15.52:0/83965689 wait complete. 2024-05-11T01:13:28.164 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2024-05-11T01:13:28.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.164 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_02_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:28.184 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:28.185 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:28.185 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_02_00_UTC 2024-05-11T01:13:28.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.185 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_02_00_UTC) 2024-05-11T01:13:28.247 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:28.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.247 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:28.311 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:28.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.311 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_04_00_UTC) 2024-05-11T01:13:28.375 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:28.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.375 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:28.439 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:28.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.439 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:28.502 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:28.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.503 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:28.565 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:28.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.566 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:28.628 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:28.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.628 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:28.692 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:28.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.692 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:28.756 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:28.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.756 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:28.820 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:28.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.820 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:28.884 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:28.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.884 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:28.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:28.948 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2024-05-11T01:13:29.009 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1m 2024-05-11T01:13:29.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.157+0000 7f7c0f693640 1 Processor -- start 2024-05-11T01:13:29.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.158+0000 7f7c0f693640 1 -- start start 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0f693640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c08145960 0x7f7c08072750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0f693640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7c08072c90 0x7f7c080d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0f693640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c080ced10 con 0x7f7c08072200 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0f693640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7c080cf920 con 0x7f7c08072c90 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0f693640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7c080ceff0 con 0x7f7c08145960 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0cc2e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7c08072c90 0x7f7c080d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0cc2e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7c08072c90 0x7f7c080d1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50206/0 (socket says 172.21.15.52:50206) 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0cc2e640 1 -- 172.21.15.52:0/1927357314 learned_addr learned my addr 172.21.15.52:0/1927357314 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0d42f640 1 --2- 172.21.15.52:0/1927357314 >> v2:172.21.15.119:3300/0 conn(0x7f7c08145960 0x7f7c08072750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:29.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0cc2e640 1 -- 172.21.15.52:0/1927357314 >> v2:172.21.15.119:3300/0 conn(0x7f7c08145960 msgr2=0x7f7c08072750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:29.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0cc2e640 1 --2- 172.21.15.52:0/1927357314 >> v2:172.21.15.119:3300/0 conn(0x7f7c08145960 0x7f7c08072750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0cc2e640 1 -- 172.21.15.52:0/1927357314 >> v1:172.21.15.119:6789/0 conn(0x7f7c08072200 legacy=0x7f7c0815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:29.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.159+0000 7f7c0cc2e640 1 -- 172.21.15.52:0/1927357314 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c080cf2d0 con 0x7f7c08072c90 2024-05-11T01:13:29.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.160+0000 7f7c0cc2e640 1 --2- 172.21.15.52:0/1927357314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08072c90 0x7f7c080d1190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7c080cfaa0 tx=0x7f7bf802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6804fe3ca1c4ef server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:29.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.160+0000 7f7bfffff640 1 -- 172.21.15.52:0/1927357314 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8037070 con 0x7f7c08072c90 2024-05-11T01:13:29.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.160+0000 7f7bfffff640 1 -- 172.21.15.52:0/1927357314 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf8032440 con 0x7f7c08072c90 2024-05-11T01:13:29.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.160+0000 7f7bfffff640 1 -- 172.21.15.52:0/1927357314 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8032810 con 0x7f7c08072c90 2024-05-11T01:13:29.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.160+0000 7f7c0f693640 1 -- 172.21.15.52:0/1927357314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08072c90 msgr2=0x7f7c080d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:29.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.160+0000 7f7c0f693640 1 --2- 172.21.15.52:0/1927357314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08072c90 0x7f7c080d1190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7c080cfaa0 tx=0x7f7bf802a130 comp rx=0 tx=0).stop 2024-05-11T01:13:29.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.161+0000 7f7c0f693640 1 -- 172.21.15.52:0/1927357314 shutdown_connections 2024-05-11T01:13:29.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.161+0000 7f7c0f693640 1 --2- 172.21.15.52:0/1927357314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08072c90 0x7f7c080d1190 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.161+0000 7f7c0f693640 1 --2- 172.21.15.52:0/1927357314 >> v2:172.21.15.119:3300/0 conn(0x7f7c08145960 0x7f7c08072750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.161+0000 7f7c0f693640 1 -- 172.21.15.52:0/1927357314 >> 172.21.15.52:0/1927357314 conn(0x7f7c0806c8d0 msgr2=0x7f7c0806cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:29.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.161+0000 7f7c0f693640 1 -- 172.21.15.52:0/1927357314 shutdown_connections 2024-05-11T01:13:29.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.161+0000 7f7c0f693640 1 -- 172.21.15.52:0/1927357314 wait complete. 2024-05-11T01:13:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.162+0000 7f7c0f693640 1 Processor -- start 2024-05-11T01:13:29.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.162+0000 7f7c0f693640 1 -- start start 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0f693640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7c08072200 0x7f7c080ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0f693640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08145960 0x7f7c08141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0f693640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c081423f0 0x7f7c081602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0f693640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0806eeb0 con 0x7f7c08145960 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0f693640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08145d40 con 0x7f7c081423f0 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0f693640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0806f030 con 0x7f7c08072200 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0cc2e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08145960 0x7f7c08141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0cc2e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08145960 0x7f7c08141eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50208/0 (socket says 172.21.15.52:50208) 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0cc2e640 1 -- 172.21.15.52:0/3546460407 learned_addr learned my addr 172.21.15.52:0/3546460407 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0d42f640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7c08072200 0x7f7c080ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0dc30640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c081423f0 0x7f7c081602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0cc2e640 1 -- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7c08072200 msgr2=0x7f7c080ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:29.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0cc2e640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7c08072200 0x7f7c080ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0cc2e640 1 -- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c081423f0 msgr2=0x7f7c081602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0cc2e640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c081423f0 0x7f7c081602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0cc2e640 1 -- 172.21.15.52:0/3546460407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c081f5060 con 0x7f7c08145960 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0cc2e640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08145960 0x7f7c08141eb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000c00 tx=0x7f7bf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7bfdffb640 1 -- 172.21.15.52:0/3546460407 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8041020 con 0x7f7c08145960 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7bfdffb640 1 -- 172.21.15.52:0/3546460407 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf80306c0 con 0x7f7c08145960 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c081f5300 con 0x7f7c08145960 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.163+0000 7f7bfdffb640 1 -- 172.21.15.52:0/3546460407 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8037040 con 0x7f7c08145960 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.164+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081f58c0 con 0x7f7c08145960 2024-05-11T01:13:29.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.165+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c080694f0 con 0x7f7c08145960 2024-05-11T01:13:29.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.165+0000 7f7bfdffb640 1 -- 172.21.15.52:0/3546460407 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f7bf8044020 con 0x7f7c08145960 2024-05-11T01:13:29.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.166+0000 7f7bfdffb640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7bec0764d0 0x7f7bec078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:29.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.166+0000 7f7bfdffb640 1 -- 172.21.15.52:0/3546460407 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5967+0+0 (secure 0 0 0) 0x7f7bf80b7d80 con 0x7f7c08145960 2024-05-11T01:13:29.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.166+0000 7f7c0d42f640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7bec0764d0 0x7f7bec078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:29.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.167+0000 7f7c0d42f640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7bec0764d0 0x7f7bec078990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7c040079a0 tx=0x7f7c04008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:29.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.170+0000 7f7bfdffb640 1 -- 172.21.15.52:0/3546460407 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7bf804b070 con 0x7f7c08145960 2024-05-11T01:13:29.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.400+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7c0806f820 con 0x7f7bec0764d0 2024-05-11T01:13:29.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.483+0000 7f7bfdffb640 1 -- 172.21.15.52:0/3546460407 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f7c0806f820 con 0x7f7bec0764d0 2024-05-11T01:13:29.482 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir1 2024-05-11T01:13:29.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7bec0764d0 msgr2=0x7f7bec078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:29.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7bec0764d0 0x7f7bec078990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7c040079a0 tx=0x7f7c04008040 comp rx=0 tx=0).stop 2024-05-11T01:13:29.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08145960 msgr2=0x7f7c08141eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:29.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08145960 0x7f7c08141eb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000c00 tx=0x7f7bf80046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 shutdown_connections 2024-05-11T01:13:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c081423f0 0x7f7c081602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f7bec0764d0 0x7f7bec078990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c08145960 0x7f7c08141eb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 --2- 172.21.15.52:0/3546460407 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7c08072200 0x7f7c080ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 >> 172.21.15.52:0/3546460407 conn(0x7f7c0806c8d0 msgr2=0x7f7c0806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 shutdown_connections 2024-05-11T01:13:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:29.485+0000 7f7c0f693640 1 -- 172.21.15.52:0/3546460407 wait complete. 2024-05-11T01:13:29.495 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2024-05-11T01:13:29.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:29.495 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_02_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:29.520 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_02_00_UTC 2024-05-11T01:13:29.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:29.520 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_02_00_UTC) 2024-05-11T01:13:29.583 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:29.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:29.583 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:29.646 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:29.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:29.646 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_04_00_UTC) 2024-05-11T01:13:29.710 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:29.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:29.710 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:29.774 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:29.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:29.774 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:29.839 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:29.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:29.839 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:29.902 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:29.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:29.903 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:29.966 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:29.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:29.966 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:30.029 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:30.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:30.029 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:30.092 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:30.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:30.092 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:30.155 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:30.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:30.156 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:30.236 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:30.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:30.236 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:30.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:30.299 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2024-05-11T01:13:30.359 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1m 2024-05-11T01:13:30.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.506+0000 7fd8e795f640 1 Processor -- start 2024-05-11T01:13:30.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.506+0000 7fd8e795f640 1 -- start start 2024-05-11T01:13:30.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.507+0000 7fd8e795f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd8e015f4d0 0x7fd8e00d50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:30.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.507+0000 7fd8e795f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e015f8b0 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.507+0000 7fd8e795f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e00b4e40 con 0x7fd8e00d3e20 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.507+0000 7fd8e795f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e00b5120 con 0x7fd8e015f4d0 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.507+0000 7fd8e4efa640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd8e00b5ee0 0x7fd8e00d3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55382/0 (socket says 172.21.15.52:55382) 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.507+0000 7fd8e4efa640 1 -- 172.21.15.52:0/1898600056 learned_addr learned my addr 172.21.15.52:0/1898600056 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.507+0000 7fd8e56fb640 1 --2- 172.21.15.52:0/1898600056 >> v2:172.21.15.119:3300/0 conn(0x7fd8e015f4d0 0x7fd8e00d50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.508+0000 7fd8d7fff640 1 -- 172.21.15.52:0/1898600056 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260291892 0 0) 0x7fd8e015f8b0 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.508+0000 7fd8d7fff640 1 -- 172.21.15.52:0/1898600056 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8cc003520 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.508+0000 7fd8e56fb640 1 -- 172.21.15.52:0/1898600056 >> v1:172.21.15.119:6789/0 conn(0x7fd8e00d3e20 legacy=0x7fd8e00cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.508+0000 7fd8d7fff640 1 -- 172.21.15.52:0/1898600056 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1384220903 0 0) 0x7fd8e00b4e40 con 0x7fd8e00d3e20 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.508+0000 7fd8e56fb640 1 -- 172.21.15.52:0/1898600056 >> v1:172.21.15.52:6789/0 conn(0x7fd8e00b5ee0 legacy=0x7fd8e00d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.508+0000 7fd8e56fb640 1 -- 172.21.15.52:0/1898600056 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e00b5400 con 0x7fd8e015f4d0 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8e56fb640 1 --2- 172.21.15.52:0/1898600056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e015f4d0 0x7fd8e00d50c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0097a0 tx=0x7fd8dc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75d010048658afe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8d7fff640 1 -- 172.21.15.52:0/1898600056 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc037070 con 0x7fd8e015f4d0 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8d7fff640 1 -- 172.21.15.52:0/1898600056 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc02ac30 con 0x7fd8e015f4d0 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8d7fff640 1 -- 172.21.15.52:0/1898600056 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00016f0 con 0x7fd8e015f4d0 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8e795f640 1 -- 172.21.15.52:0/1898600056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e015f4d0 msgr2=0x7fd8e00d50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:30.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8e795f640 1 --2- 172.21.15.52:0/1898600056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e015f4d0 0x7fd8e00d50c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0097a0 tx=0x7fd8dc02a1a0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8e795f640 1 -- 172.21.15.52:0/1898600056 shutdown_connections 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8e795f640 1 --2- 172.21.15.52:0/1898600056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e015f4d0 0x7fd8e00d50c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8e795f640 1 -- 172.21.15.52:0/1898600056 >> 172.21.15.52:0/1898600056 conn(0x7fd8e008ebc0 msgr2=0x7fd8e009e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8e795f640 1 -- 172.21.15.52:0/1898600056 shutdown_connections 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.509+0000 7fd8e795f640 1 -- 172.21.15.52:0/1898600056 wait complete. 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e795f640 1 Processor -- start 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e795f640 1 -- start start 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e795f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00b5ee0 0x7fd8e00b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e795f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e00d3e20 0x7fd8e00cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e795f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8e015f4d0 0x7fd8e00b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e795f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e00d5600 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e795f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e00b59c0 con 0x7fd8e00d3e20 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e795f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e00d5780 con 0x7fd8e015f4d0 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e56fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00b5ee0 0x7fd8e00b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e56fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00b5ee0 0x7fd8e00b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50224/0 (socket says 172.21.15.52:50224) 2024-05-11T01:13:30.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.511+0000 7fd8e56fb640 1 -- 172.21.15.52:0/3745535489 learned_addr learned my addr 172.21.15.52:0/3745535489 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e5efc640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8e015f4d0 0x7fd8e00b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e4efa640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e00d3e20 0x7fd8e00cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e56fb640 1 -- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8e015f4d0 msgr2=0x7fd8e00b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e56fb640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8e015f4d0 0x7fd8e00b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e56fb640 1 -- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e00d3e20 msgr2=0x7fd8e00cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e56fb640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e00d3e20 0x7fd8e00cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e56fb640 1 -- 172.21.15.52:0/3745535489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e015b220 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e56fb640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00b5ee0 0x7fd8e00b74f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc035040 tx=0x7fd8dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8d5ffb640 1 -- 172.21.15.52:0/3745535489 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc037040 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8d5ffb640 1 -- 172.21.15.52:0/3745535489 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc004030 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e00cc380 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8d5ffb640 1 -- 172.21.15.52:0/3745535489 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc004400 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.512+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e00cc940 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.513+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e00694f0 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.514+0000 7fd8d5ffb640 1 -- 172.21.15.52:0/3745535489 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fd8dc03a040 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.515+0000 7fd8d5ffb640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd8c40764d0 0x7fd8c4078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:30.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.515+0000 7fd8d5ffb640 1 -- 172.21.15.52:0/3745535489 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 6311+0+0 (secure 0 0 0) 0x7fd8dc0b80d0 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.515+0000 7fd8e4efa640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd8c40764d0 0x7fd8c4078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:30.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.515+0000 7fd8e4efa640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd8c40764d0 0x7fd8c4078990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8e006eee0 tx=0x7fd8d00017a0 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:30.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.519+0000 7fd8d5ffb640 1 -- 172.21.15.52:0/3745535489 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd8dc042020 con 0x7fd8e00b5ee0 2024-05-11T01:13:30.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.742+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd8e00b9920 con 0x7fd8c40764d0 2024-05-11T01:13:30.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.821+0000 7fd8d5ffb640 1 -- 172.21.15.52:0/3745535489 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fd8e00b9920 con 0x7fd8c40764d0 2024-05-11T01:13:30.821 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir2 2024-05-11T01:13:30.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd8c40764d0 msgr2=0x7fd8c4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:30.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd8c40764d0 0x7fd8c4078990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8e006eee0 tx=0x7fd8d00017a0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00b5ee0 msgr2=0x7fd8e00b74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:30.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00b5ee0 0x7fd8e00b74f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc035040 tx=0x7fd8dc0046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 shutdown_connections 2024-05-11T01:13:30.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8e015f4d0 0x7fd8e00b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e00d3e20 0x7fd8e00cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd8c40764d0 0x7fd8c4078990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 --2- 172.21.15.52:0/3745535489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00b5ee0 0x7fd8e00b74f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 >> 172.21.15.52:0/3745535489 conn(0x7fd8e008ebc0 msgr2=0x7fd8e00b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 shutdown_connections 2024-05-11T01:13:30.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:30.824+0000 7fd8e795f640 1 -- 172.21.15.52:0/3745535489 wait complete. 2024-05-11T01:13:30.833 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2024-05-11T01:13:30.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:30.834 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_02_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:30.858 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:30.859 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_02_00_UTC 2024-05-11T01:13:30.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:30.859 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_02_00_UTC) 2024-05-11T01:13:30.921 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:30.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:30.921 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:30.984 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:30.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:30.984 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_04_00_UTC) 2024-05-11T01:13:31.048 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:31.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.048 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:31.113 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:31.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.113 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:31.184 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:31.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.184 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:31.247 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:31.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.247 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:31.318 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:31.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.319 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:31.382 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:31.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.382 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:31.445 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:31.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.446 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:31.509 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:31.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.509 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:31.575 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:31.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.575 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:31.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:31.639 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2024-05-11T01:13:31.700 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1m 2024-05-11T01:13:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.851+0000 7fd8a635f640 1 Processor -- start 2024-05-11T01:13:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.851+0000 7fd8a635f640 1 -- start start 2024-05-11T01:13:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.852+0000 7fd8a635f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd8a00d3d20 0x7fd8a00d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.852+0000 7fd8a635f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd8a00d2e10 0x7fd8a00b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.852+0000 7fd8a635f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a015f8b0 con 0x7fd8a015f4d0 2024-05-11T01:13:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.852+0000 7fd8a635f640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd8a00b8920 con 0x7fd8a00d2e10 2024-05-11T01:13:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.852+0000 7fd8a635f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd8a00b8c00 con 0x7fd8a00d3d20 2024-05-11T01:13:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.852+0000 7fd8a48fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd8a00d2e10 0x7fd8a00b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.852+0000 7fd8a48fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd8a00d2e10 0x7fd8a00b99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38264/0 (socket says 172.21.15.52:38264) 2024-05-11T01:13:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.852+0000 7fd8a48fc640 1 -- 172.21.15.52:0/192816217 learned_addr learned my addr 172.21.15.52:0/192816217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.853+0000 7fd89f7fe640 1 --2- 172.21.15.52:0/192816217 >> v2:172.21.15.119:3300/0 conn(0x7fd8a00d3d20 0x7fd8a00d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.853+0000 7fd8a48fc640 1 -- 172.21.15.52:0/192816217 >> v2:172.21.15.119:3300/0 conn(0x7fd8a00d3d20 msgr2=0x7fd8a00d5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.853+0000 7fd8a48fc640 1 --2- 172.21.15.52:0/192816217 >> v2:172.21.15.119:3300/0 conn(0x7fd8a00d3d20 0x7fd8a00d5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.853+0000 7fd8a48fc640 1 -- 172.21.15.52:0/192816217 >> v1:172.21.15.119:6789/0 conn(0x7fd8a015f4d0 legacy=0x7fd8a00d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.853+0000 7fd8a48fc640 1 -- 172.21.15.52:0/192816217 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a00b8ee0 con 0x7fd8a00d2e10 2024-05-11T01:13:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.853+0000 7fd89f7fe640 1 --2- 172.21.15.52:0/192816217 >> v2:172.21.15.119:3300/0 conn(0x7fd8a00d3d20 0x7fd8a00d5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:31.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.853+0000 7fd8a48fc640 1 --2- 172.21.15.52:0/192816217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 0x7fd8a00b99c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd8a008de90 tx=0x7fd89802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d6281dc86faa1e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:31.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd89effd640 1 -- 172.21.15.52:0/192816217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898036070 con 0x7fd8a00d2e10 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd89effd640 1 -- 172.21.15.52:0/192816217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd898031600 con 0x7fd8a00d2e10 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd89effd640 1 -- 172.21.15.52:0/192816217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8980319d0 con 0x7fd8a00d2e10 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd8a635f640 1 -- 172.21.15.52:0/192816217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 msgr2=0x7fd8a00b99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd8a635f640 1 --2- 172.21.15.52:0/192816217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 0x7fd8a00b99c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd8a008de90 tx=0x7fd89802a6a0 comp rx=0 tx=0).stop 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd8a635f640 1 -- 172.21.15.52:0/192816217 shutdown_connections 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd8a635f640 1 --2- 172.21.15.52:0/192816217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 0x7fd8a00b99c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd8a635f640 1 --2- 172.21.15.52:0/192816217 >> v2:172.21.15.119:3300/0 conn(0x7fd8a00d3d20 0x7fd8a00d5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd8a635f640 1 -- 172.21.15.52:0/192816217 >> 172.21.15.52:0/192816217 conn(0x7fd8a009e450 msgr2=0x7fd8a00d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd8a635f640 1 -- 172.21.15.52:0/192816217 shutdown_connections 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.854+0000 7fd8a635f640 1 -- 172.21.15.52:0/192816217 wait complete. 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.855+0000 7fd8a635f640 1 Processor -- start 2024-05-11T01:13:31.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd8a635f640 1 -- start start 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd8a635f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 0x7fd8a00cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd8a635f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8a00d3d20 0x7fd8a01f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd8a635f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8a015f4d0 0x7fd8a01f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd8a635f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00d3240 con 0x7fd8a00d2e10 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd8a635f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00b94a0 con 0x7fd8a00d3d20 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd8a635f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a00b9f00 con 0x7fd8a015f4d0 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd89ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 0x7fd8a00cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd89ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 0x7fd8a00cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38278/0 (socket says 172.21.15.52:38278) 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.856+0000 7fd89ffff640 1 -- 172.21.15.52:0/2656610172 learned_addr learned my addr 172.21.15.52:0/2656610172 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89f7fe640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8a00d3d20 0x7fd8a01f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89ffff640 1 -- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8a015f4d0 msgr2=0x7fd8a01f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89ffff640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8a015f4d0 0x7fd8a01f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89ffff640 1 -- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8a00d3d20 msgr2=0x7fd8a01f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89ffff640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8a00d3d20 0x7fd8a01f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89ffff640 1 -- 172.21.15.52:0/2656610172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a00ccdf0 con 0x7fd8a00d2e10 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89ffff640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 0x7fd8a00cade0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b3f0 tx=0x7fd89000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89cff9640 1 -- 172.21.15.52:0/2656610172 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890015020 con 0x7fd8a00d2e10 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89cff9640 1 -- 172.21.15.52:0/2656610172 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd890011830 con 0x7fd8a00d2e10 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a00cd0d0 con 0x7fd8a00d2e10 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd89cff9640 1 -- 172.21.15.52:0/2656610172 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890011c20 con 0x7fd8a00d2e10 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.857+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a00cd690 con 0x7fd8a00d2e10 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.858+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a00694f0 con 0x7fd8a00d2e10 2024-05-11T01:13:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.859+0000 7fd89cff9640 1 -- 172.21.15.52:0/2656610172 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fd890018020 con 0x7fd8a00d2e10 2024-05-11T01:13:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.859+0000 7fd89cff9640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd88c0764d0 0x7fd88c078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.859+0000 7fd89cff9640 1 -- 172.21.15.52:0/2656610172 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6487+0+0 (secure 0 0 0) 0x7fd8900979e0 con 0x7fd8a00d2e10 2024-05-11T01:13:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.860+0000 7fd89f7fe640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd88c0764d0 0x7fd88c078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.860+0000 7fd89f7fe640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd88c0764d0 0x7fd88c078990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00cf2e0 tx=0x7fd894008040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:31.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:31.864+0000 7fd89cff9640 1 -- 172.21.15.52:0/2656610172 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd89001f030 con 0x7fd8a00d2e10 2024-05-11T01:13:32.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.081+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd8a00d3960 con 0x7fd88c0764d0 2024-05-11T01:13:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.169+0000 7fd89cff9640 1 -- 172.21.15.52:0/2656610172 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fd8a00d3960 con 0x7fd88c0764d0 2024-05-11T01:13:32.168 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir3 2024-05-11T01:13:32.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd88c0764d0 msgr2=0x7fd88c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd88c0764d0 0x7fd88c078990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00cf2e0 tx=0x7fd894008040 comp rx=0 tx=0).stop 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 msgr2=0x7fd8a00cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 0x7fd8a00cade0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b3f0 tx=0x7fd89000b8b0 comp rx=0 tx=0).stop 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 shutdown_connections 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8a015f4d0 0x7fd8a01f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8a00d3d20 0x7fd8a01f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd88c0764d0 0x7fd88c078990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 --2- 172.21.15.52:0/2656610172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8a00d2e10 0x7fd8a00cade0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 >> 172.21.15.52:0/2656610172 conn(0x7fd8a009e450 msgr2=0x7fd8a00ba9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 shutdown_connections 2024-05-11T01:13:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:32.171+0000 7fd8a635f640 1 -- 172.21.15.52:0/2656610172 wait complete. 2024-05-11T01:13:32.185 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2024-05-11T01:13:32.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.185 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2024-05-11T01:13:32.224 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:32.224 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_01_UTC 2024-05-11T01:13:32.224 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:32.224 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:32.224 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:32.225 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:32.225 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:32.225 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:32.225 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:32.225 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:32.225 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:32.225 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:32.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.225 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:32.288 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_04_01_UTC 2024-05-11T01:13:32.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.289 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_04_01_UTC) 2024-05-11T01:13:32.352 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:32.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.353 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:32.421 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:32.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.421 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:32.485 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:32.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.485 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:32.549 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:32.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.549 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:32.613 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:32.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.614 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:32.677 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:32.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.677 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:32.741 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:32.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.741 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:32.806 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:32.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.806 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:32.869 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:32.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.870 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:32.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:32.933 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2024-05-11T01:13:32.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1m 2024-05-11T01:13:33.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.140+0000 7f3ccd43e640 1 Processor -- start 2024-05-11T01:13:33.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.140+0000 7f3ccd43e640 1 -- start start 2024-05-11T01:13:33.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.141+0000 7f3ccd43e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3cc80d3d20 0x7f3cc80d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:33.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.141+0000 7f3ccd43e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cc80d2e10 0x7f3cc80b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:33.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.141+0000 7f3ccd43e640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc815f8b0 con 0x7f3cc815f4d0 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.141+0000 7f3ccd43e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f3cc80b8920 con 0x7f3cc80d3d20 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.141+0000 7f3ccd43e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc80b8c00 con 0x7f3cc80d2e10 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.141+0000 7f3cc6ffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f3cc815f4d0 0x7f3cc80d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52602/0 (socket says 172.21.15.52:52602) 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.141+0000 7f3cc6ffd640 1 -- 172.21.15.52:0/947379545 learned_addr learned my addr 172.21.15.52:0/947379545 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc67fc640 1 --2- 172.21.15.52:0/947379545 >> v2:172.21.15.119:3300/0 conn(0x7f3cc80d3d20 0x7f3cc80d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc77fe640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cc80d2e10 0x7f3cc80b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3612810841 0 0) 0x7f3cc815f8b0 con 0x7f3cc815f4d0 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cb4003520 con 0x7f3cc815f4d0 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1299855494 0 0) 0x7f3cb4003520 con 0x7f3cc815f4d0 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cc80d2e10 msgr2=0x7f3cc80b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cc80d2e10 0x7f3cc80b99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 >> v2:172.21.15.119:3300/0 conn(0x7f3cc80d3d20 msgr2=0x7f3cc80d5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 --2- 172.21.15.52:0/947379545 >> v2:172.21.15.119:3300/0 conn(0x7f3cc80d3d20 0x7f3cc80d5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc80b8ee0 con 0x7f3cc815f4d0 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f3cc0001a70 con 0x7f3cc815f4d0 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 >> v1:172.21.15.52:6789/0 conn(0x7f3cc815f4d0 legacy=0x7f3cc80d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cb4003a00 0x7f3cb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cc80d2e10 0x7f3cb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb4004d40 0x7f3cb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc80b8920 con 0x7f3cb4004d40 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc80b8c00 con 0x7f3cb4003a00 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb400b9c0 con 0x7f3cc80d2e10 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc77fe640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb4004d40 0x7f3cb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc6ffd640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cb4003a00 0x7f3cb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc67fc640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cc80d2e10 0x7f3cb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc77fe640 1 -- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cc80d2e10 msgr2=0x7f3cb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:33.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc77fe640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cc80d2e10 0x7f3cb4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc77fe640 1 -- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cb4003a00 msgr2=0x7f3cb4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:33.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc77fe640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cb4003a00 0x7f3cb4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.142+0000 7f3cc77fe640 1 -- 172.21.15.52:0/947379545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb4003520 con 0x7f3cb4004d40 2024-05-11T01:13:33.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.143+0000 7f3cc77fe640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb4004d40 0x7f3cb40051c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009ac0 tx=0x7f3cb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86db93b55843b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:33.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.143+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8036070 con 0x7f3cb4004d40 2024-05-11T01:13:33.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.143+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb8031400 con 0x7f3cb4004d40 2024-05-11T01:13:33.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.143+0000 7f3cc5ffb640 1 -- 172.21.15.52:0/947379545 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8009690 con 0x7f3cb4004d40 2024-05-11T01:13:33.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.144+0000 7f3ccd43e640 1 -- 172.21.15.52:0/947379545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb4004d40 msgr2=0x7f3cb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:33.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.144+0000 7f3ccd43e640 1 --2- 172.21.15.52:0/947379545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cb4004d40 0x7f3cb40051c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009ac0 tx=0x7f3cb4005830 comp rx=0 tx=0).stop 2024-05-11T01:13:33.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.144+0000 7f3cc77fe640 1 -- 172.21.15.52:0/947379545 reap_dead start 2024-05-11T01:13:33.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.144+0000 7f3ccd43e640 1 -- 172.21.15.52:0/947379545 shutdown_connections 2024-05-11T01:13:33.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.144+0000 7f3ccd43e640 1 -- 172.21.15.52:0/947379545 >> 172.21.15.52:0/947379545 conn(0x7f3cc809e450 msgr2=0x7f3cc80d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:33.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.144+0000 7f3ccd43e640 1 -- 172.21.15.52:0/947379545 shutdown_connections 2024-05-11T01:13:33.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.144+0000 7f3ccd43e640 1 -- 172.21.15.52:0/947379545 wait complete. 2024-05-11T01:13:33.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.145+0000 7f3ccd43e640 1 Processor -- start 2024-05-11T01:13:33.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.145+0000 7f3ccd43e640 1 -- start start 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3ccd43e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cb4004d40 0x7f3cc814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3ccd43e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cc814dac0 0x7f3cc814df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3ccd43e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cc814cbf0 0x7f3cc814d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3ccd43e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc80d3280 con 0x7f3cc814cbf0 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3ccd43e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc80b94a0 con 0x7f3cc814dac0 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3ccd43e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc80b99c0 con 0x7f3cb4004d40 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cc814cbf0 0x7f3cc814d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cc814cbf0 0x7f3cc814d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38298/0 (socket says 172.21.15.52:38298) 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc77fe640 1 -- 172.21.15.52:0/1733184775 learned_addr learned my addr 172.21.15.52:0/1733184775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc67fc640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cc814dac0 0x7f3cc814df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc6ffd640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cb4004d40 0x7f3cc814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc77fe640 1 -- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cb4004d40 msgr2=0x7f3cc814fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:33.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc77fe640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cb4004d40 0x7f3cc814fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc77fe640 1 -- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cc814dac0 msgr2=0x7f3cc814df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc77fe640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cc814dac0 0x7f3cc814df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.146+0000 7f3cc77fe640 1 -- 172.21.15.52:0/1733184775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc814e830 con 0x7f3cc814cbf0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.147+0000 7f3cc77fe640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cc814cbf0 0x7f3cc814d070 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009be0 tx=0x7f3cb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.147+0000 7f3cbf7fe640 1 -- 172.21.15.52:0/1733184775 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8036040 con 0x7f3cc814cbf0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.147+0000 7f3cbf7fe640 1 -- 172.21.15.52:0/1733184775 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb8041070 con 0x7f3cc814cbf0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.147+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc814eb10 con 0x7f3cc814cbf0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.147+0000 7f3cbf7fe640 1 -- 172.21.15.52:0/1733184775 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8009690 con 0x7f3cc814cbf0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.147+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc814f0d0 con 0x7f3cc814cbf0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.148+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc80694f0 con 0x7f3cc814cbf0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.148+0000 7f3cbf7fe640 1 -- 172.21.15.52:0/1733184775 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f3cb8039070 con 0x7f3cc814cbf0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.149+0000 7f3cbf7fe640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ca8076500 0x7f3ca80789c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.149+0000 7f3cbf7fe640 1 -- 172.21.15.52:0/1733184775 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6503+0+0 (secure 0 0 0) 0x7f3cb80b7360 con 0x7f3cc814cbf0 2024-05-11T01:13:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.149+0000 7f3cc6ffd640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ca8076500 0x7f3ca80789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:33.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.149+0000 7f3cc6ffd640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ca8076500 0x7f3ca80789c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00012e0 tx=0x7f3cc0006040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:33.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.153+0000 7f3cbf7fe640 1 -- 172.21.15.52:0/1733184775 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3cb804a020 con 0x7f3cc814cbf0 2024-05-11T01:13:33.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.414+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3cc80d5e20 con 0x7f3ca8076500 2024-05-11T01:13:33.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.505+0000 7f3cbf7fe640 1 -- 172.21.15.52:0/1733184775 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f3cc80d5e20 con 0x7f3ca8076500 2024-05-11T01:13:33.504 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir4 2024-05-11T01:13:33.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ca8076500 msgr2=0x7f3ca80789c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:33.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ca8076500 0x7f3ca80789c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00012e0 tx=0x7f3cc0006040 comp rx=0 tx=0).stop 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cc814cbf0 msgr2=0x7f3cc814d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cc814cbf0 0x7f3cc814d070 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009be0 tx=0x7f3cb8004730 comp rx=0 tx=0).stop 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 shutdown_connections 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ca8076500 0x7f3ca80789c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cc814cbf0 0x7f3cc814d070 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cc814dac0 0x7f3cc814df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 --2- 172.21.15.52:0/1733184775 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3cb4004d40 0x7f3cc814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 >> 172.21.15.52:0/1733184775 conn(0x7f3cc809e450 msgr2=0x7f3cc80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 shutdown_connections 2024-05-11T01:13:33.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:33.507+0000 7f3ccd43e640 1 -- 172.21.15.52:0/1733184775 wait complete. 2024-05-11T01:13:33.517 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2024-05-11T01:13:33.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:33.517 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2024-05-11T01:13:33.538 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:33.538 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:33.538 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:33.538 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:33.538 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:33.538 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:33.539 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:33.539 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:33.539 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:33.539 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:33.539 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:33.539 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:33.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:33.539 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:33.603 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:33.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:33.603 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_04_00_UTC) 2024-05-11T01:13:33.667 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:33.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:33.667 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:33.732 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:33.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:33.732 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:33.797 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:33.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:33.797 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:33.862 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:33.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:33.862 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:33.927 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:33.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:33.927 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:33.990 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:33.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:33.990 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:34.054 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:34.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:34.054 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:34.117 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:34.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:34.117 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:34.192 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:34.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:34.192 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:34.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:34.266 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2024-05-11T01:13:34.327 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1m 2024-05-11T01:13:34.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.490+0000 7f945ead1640 1 Processor -- start 2024-05-11T01:13:34.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.491+0000 7f945ead1640 1 -- start start 2024-05-11T01:13:34.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.491+0000 7f945ead1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f945815f4d0 0x7f94580d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.491+0000 7f945ead1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94580d2e10 0x7f94580d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.491+0000 7f945ead1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945815f8b0 con 0x7f94580d3b50 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.491+0000 7f945ead1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f94580b8920 con 0x7f945815f4d0 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.491+0000 7f945ead1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f94580b8c00 con 0x7f94580d2e10 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f945d06e640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f94580d3b50 0x7f94580cfcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52618/0 (socket says 172.21.15.52:52618) 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f945d06e640 1 -- 172.21.15.52:0/3686656523 learned_addr learned my addr 172.21.15.52:0/3686656523 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f945c86d640 1 --2- 172.21.15.52:0/3686656523 >> v2:172.21.15.119:3300/0 conn(0x7f945815f4d0 0x7f94580d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f9457fff640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94580d2e10 0x7f94580d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3255243604 0 0) 0x7f945815f8b0 con 0x7f94580d3b50 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9444003520 con 0x7f94580d3b50 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 502401328 0 0) 0x7f9444003520 con 0x7f94580d3b50 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94580d2e10 msgr2=0x7f94580d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:34.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94580d2e10 0x7f94580d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 >> v2:172.21.15.119:3300/0 conn(0x7f945815f4d0 msgr2=0x7f94580d5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 --2- 172.21.15.52:0/3686656523 >> v2:172.21.15.119:3300/0 conn(0x7f945815f4d0 0x7f94580d5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94580b8ee0 con 0x7f94580d3b50 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f94500016f0 con 0x7f94580d3b50 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f945c86d640 1 --2- 172.21.15.52:0/3686656523 >> v2:172.21.15.119:3300/0 conn(0x7f945815f4d0 0x7f94580d5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 >> v1:172.21.15.52:6789/0 conn(0x7f94580d3b50 legacy=0x7f94580cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9444003a00 0x7f9444003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444004800 0x7f9444004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94580d2e10 0x7f94440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94580b8920 con 0x7f9444003a00 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94580b8c00 con 0x7f9444004800 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f944400b9c0 con 0x7f94580d2e10 2024-05-11T01:13:34.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f945c86d640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9444003a00 0x7f9444003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.492+0000 7f9457fff640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444004800 0x7f9444004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f945d06e640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94580d2e10 0x7f94440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f945c86d640 1 -- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94580d2e10 msgr2=0x7f94440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f945c86d640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94580d2e10 0x7f94440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f945c86d640 1 -- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444004800 msgr2=0x7f9444004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f945c86d640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444004800 0x7f9444004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f945c86d640 1 -- 172.21.15.52:0/3686656523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9444003520 con 0x7f9444003a00 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f9457fff640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444004800 0x7f9444004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f945c86d640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9444003a00 0x7f9444003e20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9440009cc0 tx=0x7f9444005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde54709b7a376de server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440036070 con 0x7f9444003a00 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.493+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9440031400 con 0x7f9444003a00 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.494+0000 7f94577fe640 1 -- 172.21.15.52:0/3686656523 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440009860 con 0x7f9444003a00 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.494+0000 7f945ead1640 1 -- 172.21.15.52:0/3686656523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9444003a00 msgr2=0x7f9444003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:34.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.494+0000 7f945ead1640 1 --2- 172.21.15.52:0/3686656523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9444003a00 0x7f9444003e20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9440009cc0 tx=0x7f9444005830 comp rx=0 tx=0).stop 2024-05-11T01:13:34.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.494+0000 7f945d06e640 1 -- 172.21.15.52:0/3686656523 reap_dead start 2024-05-11T01:13:34.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.494+0000 7f945ead1640 1 -- 172.21.15.52:0/3686656523 shutdown_connections 2024-05-11T01:13:34.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.494+0000 7f945ead1640 1 -- 172.21.15.52:0/3686656523 >> 172.21.15.52:0/3686656523 conn(0x7f945809e450 msgr2=0x7f94580d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:34.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.494+0000 7f945ead1640 1 -- 172.21.15.52:0/3686656523 shutdown_connections 2024-05-11T01:13:34.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.494+0000 7f945ead1640 1 -- 172.21.15.52:0/3686656523 wait complete. 2024-05-11T01:13:34.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.495+0000 7f945ead1640 1 Processor -- start 2024-05-11T01:13:34.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.495+0000 7f945ead1640 1 -- start start 2024-05-11T01:13:34.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f945ead1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444003a00 0x7f945815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f945ead1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94581f54f0 0x7f94580a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f945ead1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94581f5950 0x7f94580a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f945ead1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94580d3270 con 0x7f94581f54f0 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f945ead1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94580b94a0 con 0x7f9444003a00 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f945ead1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f94580ba9d0 con 0x7f94581f5950 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f9457fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94581f54f0 0x7f94580a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f9457fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94581f54f0 0x7f94580a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38328/0 (socket says 172.21.15.52:38328) 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f9457fff640 1 -- 172.21.15.52:0/1276890926 learned_addr learned my addr 172.21.15.52:0/1276890926 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f945d06e640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94581f5950 0x7f94580a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.496+0000 7f945c86d640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444003a00 0x7f945815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f9457fff640 1 -- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94581f5950 msgr2=0x7f94580a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f9457fff640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94581f5950 0x7f94580a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f9457fff640 1 -- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444003a00 msgr2=0x7f945815acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:34.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f9457fff640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444003a00 0x7f945815acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f9457fff640 1 -- 172.21.15.52:0/1276890926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94581f5e40 con 0x7f94581f54f0 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f945d06e640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94581f5950 0x7f94580a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f9457fff640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94581f54f0 0x7f94580a1040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f94480101f0 tx=0x7f94480106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f94557fa640 1 -- 172.21.15.52:0/1276890926 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944801a030 con 0x7f94581f54f0 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f94557fa640 1 -- 172.21.15.52:0/1276890926 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9448010e80 con 0x7f94581f54f0 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94580cddd0 con 0x7f94581f54f0 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f94557fa640 1 -- 172.21.15.52:0/1276890926 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448003bb0 con 0x7f94581f54f0 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.497+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94580a0840 con 0x7f94581f54f0 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.498+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94580694f0 con 0x7f94581f54f0 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.499+0000 7f94557fa640 1 -- 172.21.15.52:0/1276890926 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f944801d070 con 0x7f94581f54f0 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.499+0000 7f94557fa640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f94380764d0 0x7f9438078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:34.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.499+0000 7f94557fa640 1 -- 172.21.15.52:0/1276890926 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6871+0+0 (secure 0 0 0) 0x7f9448098d70 con 0x7f94581f54f0 2024-05-11T01:13:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.499+0000 7f945c86d640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f94380764d0 0x7f9438078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.500+0000 7f945c86d640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f94380764d0 0x7f9438078990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9440033000 tx=0x7f9440008030 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:34.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.504+0000 7f94557fa640 1 -- 172.21.15.52:0/1276890926 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9448061eb0 con 0x7f94581f54f0 2024-05-11T01:13:34.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.717+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f94580ce0b0 con 0x7f94380764d0 2024-05-11T01:13:34.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.796+0000 7f94557fa640 1 -- 172.21.15.52:0/1276890926 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f94580ce0b0 con 0x7f94380764d0 2024-05-11T01:13:34.795 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir5 2024-05-11T01:13:34.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f94380764d0 msgr2=0x7f9438078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:34.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f94380764d0 0x7f9438078990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9440033000 tx=0x7f9440008030 comp rx=0 tx=0).stop 2024-05-11T01:13:34.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94581f54f0 msgr2=0x7f94580a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:34.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94581f54f0 0x7f94580a1040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f94480101f0 tx=0x7f94480106b0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 shutdown_connections 2024-05-11T01:13:34.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f94581f5950 0x7f94580a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f94380764d0 0x7f9438078990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94581f54f0 0x7f94580a1040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 --2- 172.21.15.52:0/1276890926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9444003a00 0x7f945815acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:34.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 >> 172.21.15.52:0/1276890926 conn(0x7f945809e450 msgr2=0x7f945806d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:34.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 shutdown_connections 2024-05-11T01:13:34.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:34.798+0000 7f945ead1640 1 -- 172.21.15.52:0/1276890926 wait complete. 2024-05-11T01:13:34.810 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2024-05-11T01:13:34.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:34.810 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2024-05-11T01:13:34.834 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:34.834 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:34.835 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:34.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:34.835 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:34.900 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:34.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:34.901 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_04_00_UTC) 2024-05-11T01:13:34.965 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:34.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:34.965 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:35.059 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:35.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:35.059 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:35.081 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:35.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:35.081 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:35.144 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:35.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:35.145 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:35.208 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:35.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:35.209 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:35.271 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:35.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:35.272 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:35.335 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:35.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:35.335 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:35.398 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:35.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:35.399 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:35.462 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:35.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:35.463 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:35.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:35.526 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2024-05-11T01:13:35.585 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1m 2024-05-11T01:13:35.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.732+0000 7f17d2119640 1 Processor -- start 2024-05-11T01:13:35.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.732+0000 7f17d2119640 1 -- start start 2024-05-11T01:13:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17d2119640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc15cdd0 0x7f17cc160180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17d2119640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f17cc0d35b0 0x7f17cc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17d2119640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17cc0b2c50 con 0x7f17cc1606c0 2024-05-11T01:13:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17d2119640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f17cc0b2f30 con 0x7f17cc0d35b0 2024-05-11T01:13:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17d2119640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc0b3210 con 0x7f17cc15cdd0 2024-05-11T01:13:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17cbfff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f17cc0d35b0 0x7f17cc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:35.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17caffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f17cc1606c0 0x7f17cc0d2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52628/0 (socket says 172.21.15.52:52628) 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17caffd640 1 -- 172.21.15.52:0/2369064187 learned_addr learned my addr 172.21.15.52:0/2369064187 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17cbfff640 1 --2- 172.21.15.52:0/2369064187 >> v2:172.21.15.52:3300/0 conn(0x7f17cc0d35b0 0x7f17cc06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38336/0 (socket says 172.21.15.52:38336) 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17cb7fe640 1 --2- 172.21.15.52:0/2369064187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc15cdd0 0x7f17cc160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17cbfff640 1 -- 172.21.15.52:0/2369064187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc15cdd0 msgr2=0x7f17cc160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17cbfff640 1 --2- 172.21.15.52:0/2369064187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc15cdd0 0x7f17cc160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17cbfff640 1 -- 172.21.15.52:0/2369064187 >> v1:172.21.15.52:6789/0 conn(0x7f17cc1606c0 legacy=0x7f17cc0d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17cbfff640 1 -- 172.21.15.52:0/2369064187 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc0b34f0 con 0x7f17cc0d35b0 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17ca7fc640 1 -- 172.21.15.52:0/2369064187 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 698003787 0 0) 0x7f17cc0b2c50 con 0x7f17cc1606c0 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.733+0000 7f17cb7fe640 1 --2- 172.21.15.52:0/2369064187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc15cdd0 0x7f17cc160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:35.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.734+0000 7f17cbfff640 1 --2- 172.21.15.52:0/2369064187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 0x7f17cc06d590 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009d90 tx=0x7f17c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e6b59eeba22f5d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.734+0000 7f17ca7fc640 1 -- 172.21.15.52:0/2369064187 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4007740 con 0x7f17cc0d35b0 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.734+0000 7f17ca7fc640 1 -- 172.21.15.52:0/2369064187 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c40078e0 con 0x7f17cc0d35b0 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.734+0000 7f17ca7fc640 1 -- 172.21.15.52:0/2369064187 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4007cb0 con 0x7f17cc0d35b0 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.735+0000 7f17d2119640 1 -- 172.21.15.52:0/2369064187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 msgr2=0x7f17cc06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.735+0000 7f17d2119640 1 --2- 172.21.15.52:0/2369064187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 0x7f17cc06d590 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009d90 tx=0x7f17c402a660 comp rx=0 tx=0).stop 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.735+0000 7f17d2119640 1 -- 172.21.15.52:0/2369064187 shutdown_connections 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.735+0000 7f17d2119640 1 --2- 172.21.15.52:0/2369064187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 0x7f17cc06d590 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.735+0000 7f17d2119640 1 --2- 172.21.15.52:0/2369064187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc15cdd0 0x7f17cc160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.735+0000 7f17d2119640 1 -- 172.21.15.52:0/2369064187 >> 172.21.15.52:0/2369064187 conn(0x7f17cc09ba20 msgr2=0x7f17cc09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:35.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.735+0000 7f17d2119640 1 -- 172.21.15.52:0/2369064187 shutdown_connections 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.735+0000 7f17d2119640 1 -- 172.21.15.52:0/2369064187 wait complete. 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.736+0000 7f17d2119640 1 Processor -- start 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.736+0000 7f17d2119640 1 -- start start 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17d2119640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 0x7f17cc155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17d2119640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc15cdd0 0x7f17cc1523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17d2119640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc1606c0 0x7f17cc0cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17d2119640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc15d330 con 0x7f17cc0d35b0 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17d2119640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc15d1b0 con 0x7f17cc15cdd0 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17d2119640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc0b3cf0 con 0x7f17cc1606c0 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 0x7f17cc155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:35.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 0x7f17cc155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38352/0 (socket says 172.21.15.52:38352) 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cb7fe640 1 -- 172.21.15.52:0/3785576072 learned_addr learned my addr 172.21.15.52:0/3785576072 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cbfff640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc1606c0 0x7f17cc0cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17caffd640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc15cdd0 0x7f17cc1523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cb7fe640 1 -- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc1606c0 msgr2=0x7f17cc0cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cb7fe640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc1606c0 0x7f17cc0cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cb7fe640 1 -- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc15cdd0 msgr2=0x7f17cc1523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cb7fe640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc15cdd0 0x7f17cc1523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cb7fe640 1 -- 172.21.15.52:0/3785576072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc1528e0 con 0x7f17cc0d35b0 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17caffd640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc15cdd0 0x7f17cc1523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.737+0000 7f17cbfff640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc1606c0 0x7f17cc0cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.738+0000 7f17cb7fe640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 0x7f17cc155450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00e7e0 tx=0x7f17bc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.738+0000 7f17b3fff640 1 -- 172.21.15.52:0/3785576072 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00f040 con 0x7f17cc0d35b0 2024-05-11T01:13:35.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.738+0000 7f17b3fff640 1 -- 172.21.15.52:0/3785576072 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc004030 con 0x7f17cc0d35b0 2024-05-11T01:13:35.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.738+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc0b5d60 con 0x7f17cc0d35b0 2024-05-11T01:13:35.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.738+0000 7f17b3fff640 1 -- 172.21.15.52:0/3785576072 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc004400 con 0x7f17cc0d35b0 2024-05-11T01:13:35.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.738+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc0c7b70 con 0x7f17cc0d35b0 2024-05-11T01:13:35.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.739+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc0694f0 con 0x7f17cc0d35b0 2024-05-11T01:13:35.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.739+0000 7f17b3fff640 1 -- 172.21.15.52:0/3785576072 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f17bc00c6d0 con 0x7f17cc0d35b0 2024-05-11T01:13:35.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.740+0000 7f17b3fff640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f17b80764d0 0x7f17b8078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:35.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.740+0000 7f17b3fff640 1 -- 172.21.15.52:0/3785576072 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 7043+0+0 (secure 0 0 0) 0x7f17bc097d70 con 0x7f17cc0d35b0 2024-05-11T01:13:35.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.740+0000 7f17caffd640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f17b80764d0 0x7f17b8078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:35.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.741+0000 7f17caffd640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f17b80764d0 0x7f17b8078990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f17cc15c900 tx=0x7f17c0009040 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:35.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.744+0000 7f17b3fff640 1 -- 172.21.15.52:0/3785576072 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f17cc1528e0 con 0x7f17cc0d35b0 2024-05-11T01:13:35.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:35.959+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f17cc0b2990 con 0x7f17b80764d0 2024-05-11T01:13:36.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.055+0000 7f17b3fff640 1 -- 172.21.15.52:0/3785576072 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f17cc0b2990 con 0x7f17b80764d0 2024-05-11T01:13:36.054 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir6 2024-05-11T01:13:36.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f17b80764d0 msgr2=0x7f17b8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:36.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f17b80764d0 0x7f17b8078990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f17cc15c900 tx=0x7f17c0009040 comp rx=0 tx=0).stop 2024-05-11T01:13:36.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 msgr2=0x7f17cc155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:36.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 0x7f17cc155450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00e7e0 tx=0x7f17bc00eca0 comp rx=0 tx=0).stop 2024-05-11T01:13:36.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 shutdown_connections 2024-05-11T01:13:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f17cc1606c0 0x7f17cc0cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc15cdd0 0x7f17cc1523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f17b80764d0 0x7f17b8078990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 --2- 172.21.15.52:0/3785576072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17cc0d35b0 0x7f17cc155450 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 >> 172.21.15.52:0/3785576072 conn(0x7f17cc09ba20 msgr2=0x7f17cc09bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 shutdown_connections 2024-05-11T01:13:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:36.057+0000 7f17d2119640 1 -- 172.21.15.52:0/3785576072 wait complete. 2024-05-11T01:13:36.067 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2024-05-11T01:13:36.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.067 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2024-05-11T01:13:36.091 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:36.091 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:36.091 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:36.091 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:36.091 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:36.091 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:36.091 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:36.092 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:36.092 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:36.092 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:36.092 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:36.092 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:36.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.092 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:36.156 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:36.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.156 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_04_00_UTC) 2024-05-11T01:13:36.219 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:36.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.219 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:36.283 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:36.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.283 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:36.348 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:36.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.348 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:36.414 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:36.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.414 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:36.477 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:36.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.477 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:36.541 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:36.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.541 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:36.604 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:36.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.604 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:36.667 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:36.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.667 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:36.731 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:36.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.731 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:36.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:36.795 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2024-05-11T01:13:36.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1m 2024-05-11T01:13:37.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.009+0000 7fa595d78640 1 Processor -- start 2024-05-11T01:13:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.009+0000 7fa595d78640 1 -- start start 2024-05-11T01:13:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.011+0000 7fa595d78640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa590145200 0x7fa59006f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:37.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.011+0000 7fa595d78640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5900b3ba0 0x7fa59015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:37.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.011+0000 7fa595d78640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59015f4c0 con 0x7fa59006fc90 2024-05-11T01:13:37.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.011+0000 7fa595d78640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fa59015f270 con 0x7fa590145200 2024-05-11T01:13:37.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.011+0000 7fa595d78640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa5901600d0 con 0x7fa5900b3ba0 2024-05-11T01:13:37.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.011+0000 7fa58effd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa59006fc90 0x7fa5900b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52638/0 (socket says 172.21.15.52:52638) 2024-05-11T01:13:37.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.011+0000 7fa58effd640 1 -- 172.21.15.52:0/2550478399 learned_addr learned my addr 172.21.15.52:0/2550478399 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:37.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.011+0000 7fa58ffff640 1 --2- 172.21.15.52:0/2550478399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5900b3ba0 0x7fa59015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:37.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.011+0000 7fa58f7fe640 1 --2- 172.21.15.52:0/2550478399 >> v2:172.21.15.52:3300/0 conn(0x7fa590145200 0x7fa59006f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:37.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.012+0000 7fa58e7fc640 1 -- 172.21.15.52:0/2550478399 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 616179513 0 0) 0x7fa59015f4c0 con 0x7fa59006fc90 2024-05-11T01:13:37.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.012+0000 7fa58ffff640 1 -- 172.21.15.52:0/2550478399 >> v2:172.21.15.52:3300/0 conn(0x7fa590145200 msgr2=0x7fa59006f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:37.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.012+0000 7fa58ffff640 1 --2- 172.21.15.52:0/2550478399 >> v2:172.21.15.52:3300/0 conn(0x7fa590145200 0x7fa59006f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.012+0000 7fa58ffff640 1 -- 172.21.15.52:0/2550478399 >> v1:172.21.15.52:6789/0 conn(0x7fa59006fc90 legacy=0x7fa5900b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:37.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.012+0000 7fa58ffff640 1 -- 172.21.15.52:0/2550478399 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59015f7a0 con 0x7fa5900b3ba0 2024-05-11T01:13:37.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.012+0000 7fa58f7fe640 1 --2- 172.21.15.52:0/2550478399 >> v2:172.21.15.52:3300/0 conn(0x7fa590145200 0x7fa59006f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:37.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.014+0000 7fa58ffff640 1 --2- 172.21.15.52:0/2550478399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5900b3ba0 0x7fa59015bea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa588009d90 tx=0x7fa58802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2000daf5b3b43a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:37.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.014+0000 7fa58e7fc640 1 -- 172.21.15.52:0/2550478399 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588007740 con 0x7fa5900b3ba0 2024-05-11T01:13:37.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.014+0000 7fa58e7fc640 1 -- 172.21.15.52:0/2550478399 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5880078e0 con 0x7fa5900b3ba0 2024-05-11T01:13:37.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.014+0000 7fa58e7fc640 1 -- 172.21.15.52:0/2550478399 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588007cb0 con 0x7fa5900b3ba0 2024-05-11T01:13:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.015+0000 7fa595d78640 1 -- 172.21.15.52:0/2550478399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5900b3ba0 msgr2=0x7fa59015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.015+0000 7fa595d78640 1 --2- 172.21.15.52:0/2550478399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5900b3ba0 0x7fa59015bea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa588009d90 tx=0x7fa58802a660 comp rx=0 tx=0).stop 2024-05-11T01:13:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.015+0000 7fa595d78640 1 -- 172.21.15.52:0/2550478399 shutdown_connections 2024-05-11T01:13:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.015+0000 7fa595d78640 1 --2- 172.21.15.52:0/2550478399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa5900b3ba0 0x7fa59015bea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.015+0000 7fa595d78640 1 --2- 172.21.15.52:0/2550478399 >> v2:172.21.15.52:3300/0 conn(0x7fa590145200 0x7fa59006f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.015+0000 7fa595d78640 1 -- 172.21.15.52:0/2550478399 >> 172.21.15.52:0/2550478399 conn(0x7fa590075ca0 msgr2=0x7fa5900760b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.016+0000 7fa595d78640 1 -- 172.21.15.52:0/2550478399 shutdown_connections 2024-05-11T01:13:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.016+0000 7fa595d78640 1 -- 172.21.15.52:0/2550478399 wait complete. 2024-05-11T01:13:37.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.018+0000 7fa595d78640 1 Processor -- start 2024-05-11T01:13:37.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.018+0000 7fa595d78640 1 -- start start 2024-05-11T01:13:37.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa595d78640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa59006fc90 0x7fa59013f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:37.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa595d78640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5900b3ba0 0x7fa5900ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa595d78640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa590145200 0x7fa59015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa595d78640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa590075410 con 0x7fa5900b3ba0 2024-05-11T01:13:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa595d78640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa59015fd60 con 0x7fa59006fc90 2024-05-11T01:13:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa595d78640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa590075590 con 0x7fa590145200 2024-05-11T01:13:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa59006fc90 0x7fa59013f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa58effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5900b3ba0 0x7fa5900ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa58f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa59006fc90 0x7fa59013f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.52:47340/0 (socket says 172.21.15.52:47340) 2024-05-11T01:13:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa58f7fe640 1 -- 172.21.15.52:0/985704581 learned_addr learned my addr 172.21.15.52:0/985704581 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.020+0000 7fa58ffff640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa590145200 0x7fa59015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.021+0000 7fa58f7fe640 1 -- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa590145200 msgr2=0x7fa59015b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.021+0000 7fa58f7fe640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa590145200 0x7fa59015b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.021+0000 7fa58f7fe640 1 -- 172.21.15.52:0/985704581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5900b3ba0 msgr2=0x7fa5900ab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.021+0000 7fa58f7fe640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5900b3ba0 0x7fa5900ab020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.021+0000 7fa58f7fe640 1 -- 172.21.15.52:0/985704581 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5901f9300 con 0x7fa59006fc90 2024-05-11T01:13:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.022+0000 7fa58f7fe640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa59006fc90 0x7fa59013f410 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e960 tx=0x7fa58000ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.022+0000 7fa587fff640 1 -- 172.21.15.52:0/985704581 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580017070 con 0x7fa59006fc90 2024-05-11T01:13:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.022+0000 7fa58effd640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5900b3ba0 0x7fa5900ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.023+0000 7fa587fff640 1 -- 172.21.15.52:0/985704581 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa58000ca00 con 0x7fa59006fc90 2024-05-11T01:13:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.023+0000 7fa587fff640 1 -- 172.21.15.52:0/985704581 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58000cdb0 con 0x7fa59006fc90 2024-05-11T01:13:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.023+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5901f9880 con 0x7fa59006fc90 2024-05-11T01:13:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.023+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5900b2110 con 0x7fa59006fc90 2024-05-11T01:13:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.025+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa590069c00 con 0x7fa59006fc90 2024-05-11T01:13:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.026+0000 7fa587fff640 1 -- 172.21.15.52:0/985704581 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7fa58001a070 con 0x7fa59006fc90 2024-05-11T01:13:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.027+0000 7fa587fff640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa5700764d0 0x7fa570078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.027+0000 7fa587fff640 1 -- 172.21.15.52:0/985704581 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6647+0+0 (secure 0 0 0) 0x7fa580097b20 con 0x7fa59006fc90 2024-05-11T01:13:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.027+0000 7fa58effd640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa5700764d0 0x7fa570078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:37.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.029+0000 7fa58effd640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa5700764d0 0x7fa570078990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa59015e9a0 tx=0x7fa578007cf0 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:37.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.035+0000 7fa587fff640 1 -- 172.21.15.52:0/985704581 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa580060810 con 0x7fa59006fc90 2024-05-11T01:13:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.286+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fa59006f0f0 con 0x7fa5700764d0 2024-05-11T01:13:37.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.363+0000 7fa587fff640 1 -- 172.21.15.52:0/985704581 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fa59006f0f0 con 0x7fa5700764d0 2024-05-11T01:13:37.363 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir7 2024-05-11T01:13:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.365+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa5700764d0 msgr2=0x7fa570078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.365+0000 7fa595d78640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa5700764d0 0x7fa570078990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa59015e9a0 tx=0x7fa578007cf0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.365+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa59006fc90 msgr2=0x7fa59013f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.365+0000 7fa595d78640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa59006fc90 0x7fa59013f410 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e960 tx=0x7fa58000ee20 comp rx=0 tx=0).stop 2024-05-11T01:13:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.365+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 shutdown_connections 2024-05-11T01:13:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.365+0000 7fa595d78640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa590145200 0x7fa59015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.365+0000 7fa595d78640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fa5700764d0 0x7fa570078990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.365+0000 7fa595d78640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5900b3ba0 0x7fa5900ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.365+0000 7fa595d78640 1 --2- 172.21.15.52:0/985704581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa59006fc90 0x7fa59013f410 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.366+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 >> 172.21.15.52:0/985704581 conn(0x7fa590075ca0 msgr2=0x7fa5900b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.366+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 shutdown_connections 2024-05-11T01:13:37.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:37.366+0000 7fa595d78640 1 -- 172.21.15.52:0/985704581 wait complete. 2024-05-11T01:13:37.375 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2024-05-11T01:13:37.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.376 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2024-05-11T01:13:37.399 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:37.399 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:37.399 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:37.399 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:37.399 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:37.399 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:37.400 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:37.400 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:37.400 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:37.400 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:37.400 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:37.400 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:37.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.400 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:37.463 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:37.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.463 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_04_00_UTC) 2024-05-11T01:13:37.527 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:37.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.527 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:37.590 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:37.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.591 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:37.654 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:37.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.654 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:37.717 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:37.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.717 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:37.780 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:37.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.780 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:37.843 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:37.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.843 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:37.907 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:37.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.907 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:37.970 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:37.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:37.970 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:38.059 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:38.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:38.060 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:38.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:38.087 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2024-05-11T01:13:38.148 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1m 2024-05-11T01:13:38.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.301+0000 7f3ae2c72640 1 Processor -- start 2024-05-11T01:13:38.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.301+0000 7f3ae2c72640 1 -- start start 2024-05-11T01:13:38.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae2c72640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc15cdd0 0x7f3adc160180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae2c72640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3adc0d35b0 0x7f3adc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae2c72640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3adc0b2c50 con 0x7f3adc1606c0 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae2c72640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f3adc0b2f30 con 0x7f3adc0d35b0 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae2c72640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc0b3210 con 0x7f3adc15cdd0 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae120f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3adc0d35b0 0x7f3adc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3adbfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f3adc1606c0 0x7f3adc0d2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:52642/0 (socket says 172.21.15.52:52642) 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3adbfff640 1 -- 172.21.15.52:0/305622308 learned_addr learned my addr 172.21.15.52:0/305622308 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae120f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f3adc0d35b0 0x7f3adc06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38382/0 (socket says 172.21.15.52:38382) 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae0a0e640 1 --2- 172.21.15.52:0/305622308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc15cdd0 0x7f3adc160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae120f640 1 -- 172.21.15.52:0/305622308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc15cdd0 msgr2=0x7f3adc160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae120f640 1 --2- 172.21.15.52:0/305622308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc15cdd0 0x7f3adc160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae120f640 1 -- 172.21.15.52:0/305622308 >> v1:172.21.15.52:6789/0 conn(0x7f3adc1606c0 legacy=0x7f3adc0d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:38.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3ae120f640 1 -- 172.21.15.52:0/305622308 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc0b34f0 con 0x7f3adc0d35b0 2024-05-11T01:13:38.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.302+0000 7f3adb7fe640 1 -- 172.21.15.52:0/305622308 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780454592 0 0) 0x7f3adc0b2c50 con 0x7f3adc1606c0 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.303+0000 7f3ae120f640 1 --2- 172.21.15.52:0/305622308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc0d35b0 0x7f3adc06d590 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78043a12daf245d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.303+0000 7f3adb7fe640 1 -- 172.21.15.52:0/305622308 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007740 con 0x7f3adc0d35b0 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.303+0000 7f3adb7fe640 1 -- 172.21.15.52:0/305622308 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad40078e0 con 0x7f3adc0d35b0 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.303+0000 7f3adb7fe640 1 -- 172.21.15.52:0/305622308 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007cb0 con 0x7f3adc0d35b0 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.304+0000 7f3ae2c72640 1 -- 172.21.15.52:0/305622308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc0d35b0 msgr2=0x7f3adc06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.304+0000 7f3ae2c72640 1 --2- 172.21.15.52:0/305622308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc0d35b0 0x7f3adc06d590 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad402a660 comp rx=0 tx=0).stop 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.304+0000 7f3ae2c72640 1 -- 172.21.15.52:0/305622308 shutdown_connections 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.304+0000 7f3ae2c72640 1 --2- 172.21.15.52:0/305622308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc0d35b0 0x7f3adc06d590 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.304+0000 7f3ae2c72640 1 --2- 172.21.15.52:0/305622308 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc15cdd0 0x7f3adc160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.304+0000 7f3ae2c72640 1 -- 172.21.15.52:0/305622308 >> 172.21.15.52:0/305622308 conn(0x7f3adc09ba20 msgr2=0x7f3adc09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:38.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.304+0000 7f3ae2c72640 1 -- 172.21.15.52:0/305622308 shutdown_connections 2024-05-11T01:13:38.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.304+0000 7f3ae2c72640 1 -- 172.21.15.52:0/305622308 wait complete. 2024-05-11T01:13:38.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.305+0000 7f3ae2c72640 1 Processor -- start 2024-05-11T01:13:38.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.305+0000 7f3ae2c72640 1 -- start start 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3ae2c72640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc0d35b0 0x7f3adc152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3ae2c72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc15cdd0 0x7f3adc155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3ae2c72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3adc1606c0 0x7f3adc152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3ae2c72640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc15d330 con 0x7f3adc15cdd0 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3ae2c72640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc15d1b0 con 0x7f3adc1606c0 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3ae2c72640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc0b3cf0 con 0x7f3adc0d35b0 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3adbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc15cdd0 0x7f3adc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3adbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc15cdd0 0x7f3adc155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38390/0 (socket says 172.21.15.52:38390) 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3adbfff640 1 -- 172.21.15.52:0/3281543314 learned_addr learned my addr 172.21.15.52:0/3281543314 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3ae0a0e640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc0d35b0 0x7f3adc152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:38.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.306+0000 7f3ae120f640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3adc1606c0 0x7f3adc152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3adbfff640 1 -- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc0d35b0 msgr2=0x7f3adc152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3adbfff640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc0d35b0 0x7f3adc152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3adbfff640 1 -- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3adc1606c0 msgr2=0x7f3adc152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3adbfff640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3adc1606c0 0x7f3adc152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3adbfff640 1 -- 172.21.15.52:0/3281543314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc0b8d20 con 0x7f3adc15cdd0 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3ae120f640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3adc1606c0 0x7f3adc152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3adbfff640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc15cdd0 0x7f3adc155810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800c410 tx=0x7f3ac800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3ad97fa640 1 -- 172.21.15.52:0/3281543314 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80074e0 con 0x7f3adc15cdd0 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3ad97fa640 1 -- 172.21.15.52:0/3281543314 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac80047e0 con 0x7f3adc15cdd0 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3adc0c7d80 con 0x7f3adc15cdd0 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3ad97fa640 1 -- 172.21.15.52:0/3281543314 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8004bd0 con 0x7f3adc15cdd0 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.307+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3adc0c8340 con 0x7f3adc15cdd0 2024-05-11T01:13:38.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.308+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3adc0694f0 con 0x7f3adc15cdd0 2024-05-11T01:13:38.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.309+0000 7f3ad97fa640 1 -- 172.21.15.52:0/3281543314 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f3ac8011070 con 0x7f3adc15cdd0 2024-05-11T01:13:38.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.309+0000 7f3ad97fa640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ac00764d0 0x7f3ac0078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:38.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.309+0000 7f3ad97fa640 1 -- 172.21.15.52:0/3281543314 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6711+0+0 (secure 0 0 0) 0x7f3ac8097940 con 0x7f3adc15cdd0 2024-05-11T01:13:38.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.309+0000 7f3ae0a0e640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ac00764d0 0x7f3ac0078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:38.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.310+0000 7f3ae0a0e640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ac00764d0 0x7f3ac0078990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009b40 tx=0x7f3ad0009190 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:38.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.313+0000 7f3ad97fa640 1 -- 172.21.15.52:0/3281543314 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3adc0b8d20 con 0x7f3adc15cdd0 2024-05-11T01:13:38.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.529+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3adc0b20d0 con 0x7f3ac00764d0 2024-05-11T01:13:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.610+0000 7f3ad97fa640 1 -- 172.21.15.52:0/3281543314 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f3adc0b20d0 con 0x7f3ac00764d0 2024-05-11T01:13:38.610 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir8 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.612+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ac00764d0 msgr2=0x7f3ac0078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.612+0000 7f3ae2c72640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ac00764d0 0x7f3ac0078990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009b40 tx=0x7f3ad0009190 comp rx=0 tx=0).stop 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.612+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc15cdd0 msgr2=0x7f3adc155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.612+0000 7f3ae2c72640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc15cdd0 0x7f3adc155810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800c410 tx=0x7f3ac800c8d0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.613+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 shutdown_connections 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.613+0000 7f3ae2c72640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3adc1606c0 0x7f3adc152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.613+0000 7f3ae2c72640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3ac00764d0 0x7f3ac0078990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.613+0000 7f3ae2c72640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc15cdd0 0x7f3adc155810 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.613+0000 7f3ae2c72640 1 --2- 172.21.15.52:0/3281543314 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3adc0d35b0 0x7f3adc152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.613+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 >> 172.21.15.52:0/3281543314 conn(0x7f3adc09ba20 msgr2=0x7f3adc06e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:38.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.613+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 shutdown_connections 2024-05-11T01:13:38.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:38.613+0000 7f3ae2c72640 1 -- 172.21.15.52:0/3281543314 wait complete. 2024-05-11T01:13:38.622 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2024-05-11T01:13:38.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:38.623 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2024-05-11T01:13:38.643 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:38.644 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:38.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:38.645 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:38.707 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:38.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:38.707 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_04_00_UTC) 2024-05-11T01:13:38.771 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:38.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:38.771 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:38.834 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:38.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:38.834 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:38.898 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:38.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:38.898 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:38.961 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:38.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:38.961 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:39.047 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:39.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:39.047 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:39.080 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:39.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:39.080 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:39.148 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:39.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:39.148 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:39.214 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:39.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:39.214 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:39.277 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:39.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:39.277 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:39.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:39.340 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2024-05-11T01:13:39.401 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1m 2024-05-11T01:13:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.549+0000 7f52abfe4640 1 Processor -- start 2024-05-11T01:13:39.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.550+0000 7f52abfe4640 1 -- start start 2024-05-11T01:13:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.550+0000 7f52abfe4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f52a415f4d0 0x7f52a40d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.550+0000 7f52abfe4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a40d2e10 0x7f52a40d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.550+0000 7f52abfe4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f52a40d3b50 0x7f52a40cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.550+0000 7f52abfe4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f52a415f8b0 con 0x7f52a415f4d0 2024-05-11T01:13:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.550+0000 7f52abfe4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f52a40b8920 con 0x7f52a40d3b50 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.550+0000 7f52abfe4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f52a40b8c00 con 0x7f52a40d2e10 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.550+0000 7f52a9d80640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f52a415f4d0 0x7f52a40d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52a9d80640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f52a415f4d0 0x7f52a40d5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38402/0 (socket says 172.21.15.52:38402) 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52a9d80640 1 -- 172.21.15.52:0/2317470505 learned_addr learned my addr 172.21.15.52:0/2317470505 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52a957f640 1 --2- 172.21.15.52:0/2317470505 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a40d2e10 0x7f52a40d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52aa581640 1 --2- 172.21.15.52:0/2317470505 >> v2:172.21.15.119:3300/0 conn(0x7f52a40d3b50 0x7f52a40cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52a9d80640 1 -- 172.21.15.52:0/2317470505 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a40d2e10 msgr2=0x7f52a40d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52a9d80640 1 --2- 172.21.15.52:0/2317470505 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a40d2e10 0x7f52a40d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52a9d80640 1 -- 172.21.15.52:0/2317470505 >> v2:172.21.15.119:3300/0 conn(0x7f52a40d3b50 msgr2=0x7f52a40cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52a9d80640 1 --2- 172.21.15.52:0/2317470505 >> v2:172.21.15.119:3300/0 conn(0x7f52a40d3b50 0x7f52a40cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52a9d80640 1 -- 172.21.15.52:0/2317470505 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a40b8ee0 con 0x7f52a415f4d0 2024-05-11T01:13:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.551+0000 7f52a9d80640 1 --2- 172.21.15.52:0/2317470505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a415f4d0 0x7f52a40d5a50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f52a40cf610 tx=0x7f529c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8191a366fab94139 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52a8d7e640 1 -- 172.21.15.52:0/2317470505 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c037070 con 0x7f52a415f4d0 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52a8d7e640 1 -- 172.21.15.52:0/2317470505 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f529c032440 con 0x7f52a415f4d0 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52a8d7e640 1 -- 172.21.15.52:0/2317470505 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c032810 con 0x7f52a415f4d0 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52abfe4640 1 -- 172.21.15.52:0/2317470505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a415f4d0 msgr2=0x7f52a40d5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52abfe4640 1 --2- 172.21.15.52:0/2317470505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a415f4d0 0x7f52a40d5a50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f52a40cf610 tx=0x7f529c02a1e0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52abfe4640 1 -- 172.21.15.52:0/2317470505 shutdown_connections 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52abfe4640 1 --2- 172.21.15.52:0/2317470505 >> v2:172.21.15.119:3300/0 conn(0x7f52a40d3b50 0x7f52a40cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52abfe4640 1 --2- 172.21.15.52:0/2317470505 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a40d2e10 0x7f52a40d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52abfe4640 1 --2- 172.21.15.52:0/2317470505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a415f4d0 0x7f52a40d5a50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52abfe4640 1 -- 172.21.15.52:0/2317470505 >> 172.21.15.52:0/2317470505 conn(0x7f52a409e450 msgr2=0x7f52a40d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52abfe4640 1 -- 172.21.15.52:0/2317470505 shutdown_connections 2024-05-11T01:13:39.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.552+0000 7f52abfe4640 1 -- 172.21.15.52:0/2317470505 wait complete. 2024-05-11T01:13:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.554+0000 7f52abfe4640 1 Processor -- start 2024-05-11T01:13:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.554+0000 7f52abfe4640 1 -- start start 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.554+0000 7f52abfe4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a40d2e10 0x7f52a41f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.554+0000 7f52abfe4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a40d3b50 0x7f52a41621c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.554+0000 7f52abfe4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a415f4d0 0x7f52a415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.554+0000 7f52abfe4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52a40d3270 con 0x7f52a40d3b50 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.554+0000 7f52abfe4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52a40b94a0 con 0x7f52a40d2e10 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.554+0000 7f52abfe4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f52a40d0230 con 0x7f52a415f4d0 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a957f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a40d3b50 0x7f52a41621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a957f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a40d3b50 0x7f52a41621c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38404/0 (socket says 172.21.15.52:38404) 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a957f640 1 -- 172.21.15.52:0/684562187 learned_addr learned my addr 172.21.15.52:0/684562187 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52aa581640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a415f4d0 0x7f52a415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a9d80640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a40d2e10 0x7f52a41f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:39.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a957f640 1 -- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a415f4d0 msgr2=0x7f52a415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a957f640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a415f4d0 0x7f52a415a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a957f640 1 -- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a40d2e10 msgr2=0x7f52a41f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a957f640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a40d2e10 0x7f52a41f8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a957f640 1 -- 172.21.15.52:0/684562187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a41f8d90 con 0x7f52a40d3b50 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a957f640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a40d3b50 0x7f52a41621c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f529800d800 tx=0x7f529800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a27fc640 1 -- 172.21.15.52:0/684562187 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298018020 con 0x7f52a40d3b50 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a27fc640 1 -- 172.21.15.52:0/684562187 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52980096a0 con 0x7f52a40d3b50 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a41f9070 con 0x7f52a40d3b50 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52a27fc640 1 -- 172.21.15.52:0/684562187 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298009a70 con 0x7f52a40d3b50 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.555+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a4159ec0 con 0x7f52a40d3b50 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.556+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a40694f0 con 0x7f52a40d3b50 2024-05-11T01:13:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.557+0000 7f52a27fc640 1 -- 172.21.15.52:0/684562187 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f529801b020 con 0x7f52a40d3b50 2024-05-11T01:13:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.557+0000 7f52a27fc640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f52900764d0 0x7f5290078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.557+0000 7f52a27fc640 1 -- 172.21.15.52:0/684562187 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6695+0+0 (secure 0 0 0) 0x7f5298097e90 con 0x7f52a40d3b50 2024-05-11T01:13:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.558+0000 7f52a9d80640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f52900764d0 0x7f5290078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.558+0000 7f52a9d80640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f52900764d0 0x7f5290078990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f529c034000 tx=0x7f529c02e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:39.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.562+0000 7f52a27fc640 1 -- 172.21.15.52:0/684562187 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5298022030 con 0x7f52a40d3b50 2024-05-11T01:13:39.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.804+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 --> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f52a415a2c0 con 0x7f52900764d0 2024-05-11T01:13:39.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.889+0000 7f52a27fc640 1 -- 172.21.15.52:0/684562187 <== mgr.4927 v2:172.21.15.119:6832/2051242002 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f52a415a2c0 con 0x7f52900764d0 2024-05-11T01:13:39.888 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule removed for path /snap_test_dir1/dir9 2024-05-11T01:13:39.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f52900764d0 msgr2=0x7f5290078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:39.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f52900764d0 0x7f5290078990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f529c034000 tx=0x7f529c02e000 comp rx=0 tx=0).stop 2024-05-11T01:13:39.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a40d3b50 msgr2=0x7f52a41621c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:39.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a40d3b50 0x7f52a41621c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f529800d800 tx=0x7f529800dcc0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 shutdown_connections 2024-05-11T01:13:39.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52a415f4d0 0x7f52a415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f52900764d0 0x7f5290078990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52a40d3b50 0x7f52a41621c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 --2- 172.21.15.52:0/684562187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a40d2e10 0x7f52a41f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:39.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 >> 172.21.15.52:0/684562187 conn(0x7f52a409e450 msgr2=0x7f52a406d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:39.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 shutdown_connections 2024-05-11T01:13:39.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:39.891+0000 7f52abfe4640 1 -- 172.21.15.52:0/684562187 wait complete. 2024-05-11T01:13:39.900 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2024-05-11T01:13:39.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:39.901 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2024-05-11T01:13:39.925 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:39.925 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:39.925 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:39.925 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:39.925 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:39.925 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:39.925 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:39.926 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:39.926 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:39.926 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:39.926 INFO:teuthology.orchestra.run.smithi052.stdout:scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:39.926 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_03_00_UTC 2024-05-11T01:13:39.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:39.926 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_03_00_UTC) 2024-05-11T01:13:39.989 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_04_00_UTC 2024-05-11T01:13:39.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:39.989 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_04_00_UTC) 2024-05-11T01:13:40.057 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_05_00_UTC 2024-05-11T01:13:40.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.057 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_05_00_UTC) 2024-05-11T01:13:40.120 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_06_00_UTC 2024-05-11T01:13:40.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.121 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_06_00_UTC) 2024-05-11T01:13:40.189 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_07_00_UTC 2024-05-11T01:13:40.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.189 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_07_00_UTC) 2024-05-11T01:13:40.253 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_08_00_UTC 2024-05-11T01:13:40.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.253 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_08_00_UTC) 2024-05-11T01:13:40.320 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_09_00_UTC 2024-05-11T01:13:40.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.320 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_09_00_UTC) 2024-05-11T01:13:40.382 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_10_00_UTC 2024-05-11T01:13:40.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.383 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_10_00_UTC) 2024-05-11T01:13:40.446 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_11_00_UTC 2024-05-11T01:13:40.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.446 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_11_00_UTC) 2024-05-11T01:13:40.510 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_12_00_UTC 2024-05-11T01:13:40.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.510 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_12_00_UTC) 2024-05-11T01:13:40.574 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_13_00_UTC 2024-05-11T01:13:40.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.574 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-05-11-01_13_00_UTC) 2024-05-11T01:13:40.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:40.639 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2024-05-11T01:13:40.698 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-05-11T01:13:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.846+0000 7f96f0c6d640 1 Processor -- start 2024-05-11T01:13:40.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.846+0000 7f96f0c6d640 1 -- start start 2024-05-11T01:13:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.847+0000 7f96f0c6d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec15f430 0x7f96ec0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.847+0000 7f96f0c6d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96ec0b5420 0x7f96ec0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.847+0000 7f96f0c6d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ec0cfc60 con 0x7f96ec0d1eb0 2024-05-11T01:13:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.847+0000 7f96f0c6d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f96ec0d2a80 con 0x7f96ec0b5420 2024-05-11T01:13:40.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.847+0000 7f96f0c6d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec163120 con 0x7f96ec15f430 2024-05-11T01:13:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.847+0000 7f96ea59c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec15f430 0x7f96ec0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.847+0000 7f96e9d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96ec0b5420 0x7f96ec0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.847+0000 7f96ea59c640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec15f430 0x7f96ec0b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:40308/0 (socket says 172.21.15.52:40308) 2024-05-11T01:13:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.847+0000 7f96ea59c640 1 -- 172.21.15.52:0/4243520177 learned_addr learned my addr 172.21.15.52:0/4243520177 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.848+0000 7f96e9d9b640 1 -- 172.21.15.52:0/4243520177 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec15f430 msgr2=0x7f96ec0b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.848+0000 7f96e9d9b640 1 --2- 172.21.15.52:0/4243520177 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec15f430 0x7f96ec0b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.848+0000 7f96e9d9b640 1 -- 172.21.15.52:0/4243520177 >> v1:172.21.15.119:6789/0 conn(0x7f96ec0d1eb0 legacy=0x7f96ec0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:40.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.848+0000 7f96e9d9b640 1 -- 172.21.15.52:0/4243520177 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec0d24c0 con 0x7f96ec0b5420 2024-05-11T01:13:40.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.848+0000 7f96e9d9b640 1 --2- 172.21.15.52:0/4243520177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0b5420 0x7f96ec0d1970 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f96e00097a0 tx=0x7f96e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=728160af4de17eee server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96e959a640 1 -- 172.21.15.52:0/4243520177 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0037070 con 0x7f96ec0b5420 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96e959a640 1 -- 172.21.15.52:0/4243520177 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e0032440 con 0x7f96ec0b5420 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96e959a640 1 -- 172.21.15.52:0/4243520177 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0032810 con 0x7f96ec0b5420 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96f0c6d640 1 -- 172.21.15.52:0/4243520177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0b5420 msgr2=0x7f96ec0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96f0c6d640 1 --2- 172.21.15.52:0/4243520177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0b5420 0x7f96ec0d1970 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f96e00097a0 tx=0x7f96e002a180 comp rx=0 tx=0).stop 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96f0c6d640 1 -- 172.21.15.52:0/4243520177 shutdown_connections 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96f0c6d640 1 --2- 172.21.15.52:0/4243520177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0b5420 0x7f96ec0d1970 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96f0c6d640 1 --2- 172.21.15.52:0/4243520177 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec15f430 0x7f96ec0b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96f0c6d640 1 -- 172.21.15.52:0/4243520177 >> 172.21.15.52:0/4243520177 conn(0x7f96ec08ebb0 msgr2=0x7f96ec09e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96f0c6d640 1 -- 172.21.15.52:0/4243520177 shutdown_connections 2024-05-11T01:13:40.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.849+0000 7f96f0c6d640 1 -- 172.21.15.52:0/4243520177 wait complete. 2024-05-11T01:13:40.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.851+0000 7f96f0c6d640 1 Processor -- start 2024-05-11T01:13:40.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.851+0000 7f96f0c6d640 1 -- start start 2024-05-11T01:13:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.851+0000 7f96f0c6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96ec0b5420 0x7f96ec1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.851+0000 7f96f0c6d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec0d1eb0 0x7f96ec0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.851+0000 7f96f0c6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec15f430 0x7f96ec15e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96f0c6d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0d01e0 con 0x7f96ec0b5420 2024-05-11T01:13:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96f0c6d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0cff00 con 0x7f96ec15f430 2024-05-11T01:13:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96f0c6d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec0d0e10 con 0x7f96ec0d1eb0 2024-05-11T01:13:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ea59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96ec0b5420 0x7f96ec1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:40.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ea59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96ec0b5420 0x7f96ec1609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38418/0 (socket says 172.21.15.52:38418) 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ea59c640 1 -- 172.21.15.52:0/1059835318 learned_addr learned my addr 172.21.15.52:0/1059835318 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96e9d9b640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec0d1eb0 0x7f96ec0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ead9d640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec15f430 0x7f96ec15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ea59c640 1 -- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec0d1eb0 msgr2=0x7f96ec0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ea59c640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec0d1eb0 0x7f96ec0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ea59c640 1 -- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec15f430 msgr2=0x7f96ec15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ea59c640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec15f430 0x7f96ec15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ea59c640 1 -- 172.21.15.52:0/1059835318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec1f6630 con 0x7f96ec0b5420 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96ea59c640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96ec0b5420 0x7f96ec1609b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00e840 tx=0x7f96dc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96daffd640 1 -- 172.21.15.52:0/1059835318 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc017070 con 0x7f96ec0b5420 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96daffd640 1 -- 172.21.15.52:0/1059835318 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc00c8e0 con 0x7f96ec0b5420 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec0c9770 con 0x7f96ec0b5420 2024-05-11T01:13:40.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96daffd640 1 -- 172.21.15.52:0/1059835318 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc00ccd0 con 0x7f96ec0b5420 2024-05-11T01:13:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.852+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec0c9d30 con 0x7f96ec0b5420 2024-05-11T01:13:40.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.853+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ec0695d0 con 0x7f96ec0b5420 2024-05-11T01:13:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.854+0000 7f96daffd640 1 -- 172.21.15.52:0/1059835318 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 98389+0+0 (secure 0 0 0) 0x7f96dc01a070 con 0x7f96ec0b5420 2024-05-11T01:13:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.855+0000 7f96daffd640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f96cc0764d0 0x7f96cc078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.855+0000 7f96daffd640 1 -- 172.21.15.52:0/1059835318 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6583+0+0 (secure 0 0 0) 0x7f96dc097dc0 con 0x7f96ec0b5420 2024-05-11T01:13:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.855+0000 7f96e9d9b640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f96cc0764d0 0x7f96cc078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.855+0000 7f96e9d9b640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f96cc0764d0 0x7f96cc078990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96e0034000 tx=0x7f96e002e000 comp rx=0 tx=0).ready entity=mgr.4927 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:40.859+0000 7f96daffd640 1 -- 172.21.15.52:0/1059835318 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f96dc060c20 con 0x7f96ec0b5420 2024-05-11T01:13:41.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:41.120+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f96ec0b4850 con 0x7f96ec0b5420 2024-05-11T01:13:42.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.013+0000 7f96daffd640 1 -- 172.21.15.52:0/1059835318 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v13) ==== 93+0+0 (secure 0 0 0) 0x7f96dc064f20 con 0x7f96ec0b5420 2024-05-11T01:13:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f96cc0764d0 msgr2=0x7f96cc078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f96cc0764d0 0x7f96cc078990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96e0034000 tx=0x7f96e002e000 comp rx=0 tx=0).stop 2024-05-11T01:13:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96ec0b5420 msgr2=0x7f96ec1609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96ec0b5420 0x7f96ec1609b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00e840 tx=0x7f96dc00ed00 comp rx=0 tx=0).stop 2024-05-11T01:13:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 shutdown_connections 2024-05-11T01:13:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec15f430 0x7f96ec15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f96ec0d1eb0 0x7f96ec0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f96cc0764d0 0x7f96cc078990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 --2- 172.21.15.52:0/1059835318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96ec0b5420 0x7f96ec1609b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 >> 172.21.15.52:0/1059835318 conn(0x7f96ec08ebb0 msgr2=0x7f96ec0d4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 shutdown_connections 2024-05-11T01:13:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.016+0000 7f96f0c6d640 1 -- 172.21.15.52:0/1059835318 wait complete. 2024-05-11T01:13:42.033 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T01:13:42.033 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T01:13:42.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T01:13:42.128 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T01:13:42.128 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T01:13:42.128 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:13:42.129 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:13:42.152 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:13:42.152 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:13:42.176 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:42.178+0000 7fca2c5ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T01:13:42.177 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:42.178+0000 7fbc6cbbf040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T01:13:42.193 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:13:42.289 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:42.291+0000 7fca2c5ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:13:42.298 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:42.299+0000 7fbc6cbbf040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:13:42.336 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:42.337+0000 7fca2c5ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T01:13:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.340+0000 7ff6bbfc8640 1 Processor -- start 2024-05-11T01:13:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.340+0000 7ff6bbfc8640 1 -- start start 2024-05-11T01:13:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6bbfc8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff6b415f430 0x7ff6b415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6bbfc8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b41602b0 0x7ff6b41606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6bbfc8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff6b40d1b50 0x7ff6b40d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6bbfc8640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff6b4163250 con 0x7ff6b415f430 2024-05-11T01:13:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6bbfc8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff6b4163530 con 0x7ff6b40d1b50 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6bbfc8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b4163810 con 0x7ff6b41602b0 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6b9d64640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff6b415f430 0x7ff6b415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6b9d64640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff6b415f430 0x7ff6b415fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36186/0 (socket says 172.21.15.52:36186) 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6b9d64640 1 -- 172.21.15.52:0/2377674772 learned_addr learned my addr 172.21.15.52:0/2377674772 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6ba565640 1 --2- 172.21.15.52:0/2377674772 >> v2:172.21.15.119:3300/0 conn(0x7ff6b40d1b50 0x7ff6b40d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6b9563640 1 --2- 172.21.15.52:0/2377674772 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b41602b0 0x7ff6b41606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6b9d64640 1 -- 172.21.15.52:0/2377674772 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b41602b0 msgr2=0x7ff6b41606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6b9d64640 1 --2- 172.21.15.52:0/2377674772 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b41602b0 0x7ff6b41606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6b9d64640 1 -- 172.21.15.52:0/2377674772 >> v2:172.21.15.119:3300/0 conn(0x7ff6b40d1b50 msgr2=0x7ff6b40d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6b9d64640 1 --2- 172.21.15.52:0/2377674772 >> v2:172.21.15.119:3300/0 conn(0x7ff6b40d1b50 0x7ff6b40d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.341+0000 7ff6b9d64640 1 -- 172.21.15.52:0/2377674772 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b4163af0 con 0x7ff6b415f430 2024-05-11T01:13:42.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.342+0000 7ff6b9d64640 1 --2- 172.21.15.52:0/2377674772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b415f430 0x7ff6b415fd70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40cf710 tx=0x7ff6b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3d51fa20b0013a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.342+0000 7ff6b8d62640 1 -- 172.21.15.52:0/2377674772 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0037070 con 0x7ff6b415f430 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.342+0000 7ff6b8d62640 1 -- 172.21.15.52:0/2377674772 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b0032440 con 0x7ff6b415f430 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.342+0000 7ff6b8d62640 1 -- 172.21.15.52:0/2377674772 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0032810 con 0x7ff6b415f430 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.343+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2377674772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b415f430 msgr2=0x7ff6b415fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.343+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2377674772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b415f430 0x7ff6b415fd70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40cf710 tx=0x7ff6b002a1e0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.343+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2377674772 shutdown_connections 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.343+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2377674772 >> v2:172.21.15.119:3300/0 conn(0x7ff6b40d1b50 0x7ff6b40d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.343+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2377674772 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b41602b0 0x7ff6b41606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.343+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2377674772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b415f430 0x7ff6b415fd70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.343+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2377674772 >> 172.21.15.52:0/2377674772 conn(0x7ff6b408eb70 msgr2=0x7ff6b409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.343+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2377674772 shutdown_connections 2024-05-11T01:13:42.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.343+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2377674772 wait complete. 2024-05-11T01:13:42.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.344+0000 7ff6bbfc8640 1 Processor -- start 2024-05-11T01:13:42.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.344+0000 7ff6bbfc8640 1 -- start start 2024-05-11T01:13:42.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6bbfc8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b40d1b50 0x7ff6b40ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6bbfc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6b415f430 0x7ff6b41f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6bbfc8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b41602b0 0x7ff6b40cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6bbfc8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b4163030 con 0x7ff6b41602b0 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6bbfc8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b4162eb0 con 0x7ff6b415f430 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6bbfc8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b40d2540 con 0x7ff6b40d1b50 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6ba565640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b41602b0 0x7ff6b40cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6ba565640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b41602b0 0x7ff6b40cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36200/0 (socket says 172.21.15.52:36200) 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6ba565640 1 -- 172.21.15.52:0/2324386950 learned_addr learned my addr 172.21.15.52:0/2324386950 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6b9563640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6b415f430 0x7ff6b41f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6b9d64640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b40d1b50 0x7ff6b40ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6ba565640 1 -- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b40d1b50 msgr2=0x7ff6b40ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6ba565640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b40d1b50 0x7ff6b40ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6ba565640 1 -- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6b415f430 msgr2=0x7ff6b41f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6ba565640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6b415f430 0x7ff6b41f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.345+0000 7ff6ba565640 1 -- 172.21.15.52:0/2324386950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b414af10 con 0x7ff6b41602b0 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.346+0000 7ff6ba565640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b41602b0 0x7ff6b40cb4e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00eed0 tx=0x7ff6ac00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.346+0000 7ff6aa7fc640 1 -- 172.21.15.52:0/2324386950 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac018020 con 0x7ff6b41602b0 2024-05-11T01:13:42.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.346+0000 7ff6aa7fc640 1 -- 172.21.15.52:0/2324386950 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ac014400 con 0x7ff6b41602b0 2024-05-11T01:13:42.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.346+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b41f9630 con 0x7ff6b41602b0 2024-05-11T01:13:42.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.346+0000 7ff6aa7fc640 1 -- 172.21.15.52:0/2324386950 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0147f0 con 0x7ff6b41602b0 2024-05-11T01:13:42.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.346+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b41476e0 con 0x7ff6b41602b0 2024-05-11T01:13:42.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.347+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b4069780 con 0x7ff6b41602b0 2024-05-11T01:13:42.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.347+0000 7ff6aa7fc640 1 -- 172.21.15.52:0/2324386950 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 98372+0+0 (secure 0 0 0) 0x7ff6ac005d10 con 0x7ff6b41602b0 2024-05-11T01:13:42.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.348+0000 7ff6aa7fc640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff6980764b0 0x7ff698078970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.348+0000 7ff6b9d64640 1 -- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff6980764b0 msgr2=0x7ff698078970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:42.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.348+0000 7ff6aa7fc640 1 -- 172.21.15.52:0/2324386950 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6663+0+0 (secure 0 0 0) 0x7ff6ac0982b0 con 0x7ff6b41602b0 2024-05-11T01:13:42.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.348+0000 7ff6b9d64640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff6980764b0 0x7ff698078970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:13:42.349 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:42.350+0000 7fbc6cbbf040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T01:13:42.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.352+0000 7ff6aa7fc640 1 -- 172.21.15.52:0/2324386950 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff6ac022070 con 0x7ff6b41602b0 2024-05-11T01:13:42.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.549+0000 7ff6b9d64640 1 -- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff6980764b0 msgr2=0x7ff698078970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:42.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.549+0000 7ff6b9d64640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff6980764b0 0x7ff698078970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:13:42.550 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:42.551+0000 7fca2c5ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T01:13:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.582+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6b40d2940 con 0x7ff6b41602b0 2024-05-11T01:13:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.583+0000 7ff6aa7fc640 1 -- 172.21.15.52:0/2324386950 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v46) ==== 80+0+360 (secure 0 0 0) 0x7ff6b41f9630 con 0x7ff6b41602b0 2024-05-11T01:13:42.582 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.119:0/1820176553 2024-05-12T01:01:57.869350+0000 2024-05-11T01:13:42.582 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.119:0/1584246087 2024-05-12T01:01:57.869350+0000 2024-05-11T01:13:42.582 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.119:6833/3083030853 2024-05-12T01:01:57.869350+0000 2024-05-11T01:13:42.582 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.119:6832/3083030853 2024-05-12T01:01:57.869350+0000 2024-05-11T01:13:42.582 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.119:0/1042890889 2024-05-12T01:01:57.869350+0000 2024-05-11T01:13:42.583 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.119:0/1530920757 2024-05-12T01:01:57.869350+0000 2024-05-11T01:13:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:listed 6 entries 2024-05-11T01:13:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff6980764b0 msgr2=0x7ff698078970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff6980764b0 0x7ff698078970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b41602b0 msgr2=0x7ff6b40cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b41602b0 0x7ff6b40cb4e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00eed0 tx=0x7ff6ac00b480 comp rx=0 tx=0).stop 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 shutdown_connections 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7ff6980764b0 0x7ff698078970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6b41602b0 0x7ff6b40cb4e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6b415f430 0x7ff6b41f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 --2- 172.21.15.52:0/2324386950 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6b40d1b50 0x7ff6b40ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 >> 172.21.15.52:0/2324386950 conn(0x7ff6b408eb70 msgr2=0x7ff6b40b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 shutdown_connections 2024-05-11T01:13:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.585+0000 7ff6bbfc8640 1 -- 172.21.15.52:0/2324386950 wait complete. 2024-05-11T01:13:42.589 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:42.590+0000 7fbc6cbbf040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T01:13:42.593 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:42.594+0000 7fca2c5ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T01:13:42.596 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-05-11T01:13:42.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:42.596 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-11T01:13:42.620 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[47063]: fuse finished with error 0 and tester_r 0 2024-05-11T01:13:42.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:13:42.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:13:42.622 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T01:13:42.640 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:42.641+0000 7fbc6cbbf040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T01:13:42.770 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T01:13:42.770 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:13:42.770 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:13:42.786 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13678 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T01:13:42.787 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T01:13:42.787 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:13:42.787 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:13:42.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3484 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T01:13:42.827 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T01:13:42.827 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:13:42.827 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:13:42.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13678 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T01:13:42.843 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:13:42.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.973+0000 7f462d79d640 1 Processor -- start 2024-05-11T01:13:42.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.973+0000 7f462d79d640 1 -- start start 2024-05-11T01:13:42.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f462d79d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f46280737d0 0x7f4628166b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f462d79d640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46280aea90 con 0x7f4628072ce0 2024-05-11T01:13:42.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f462d79d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46280aed70 con 0x7f46280afb30 2024-05-11T01:13:42.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f462d79d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f462806e760 con 0x7f46280737d0 2024-05-11T01:13:42.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f4626ffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f4628072ce0 0x7f462806e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36326/0 (socket says 172.21.15.52:36326) 2024-05-11T01:13:42.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f4626ffd640 1 -- 172.21.15.52:0/4215164770 learned_addr learned my addr 172.21.15.52:0/4215164770 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:42.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f46277fe640 1 --2- 172.21.15.52:0/4215164770 >> v2:172.21.15.52:3300/0 conn(0x7f46280737d0 0x7f4628166b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f46277fe640 1 -- 172.21.15.52:0/4215164770 >> v1:172.21.15.119:6789/0 conn(0x7f46280afb30 legacy=0x7f46280730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f46277fe640 1 -- 172.21.15.52:0/4215164770 >> v1:172.21.15.52:6789/0 conn(0x7f4628072ce0 legacy=0x7f462806e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.974+0000 7f46277fe640 1 -- 172.21.15.52:0/4215164770 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46280af050 con 0x7f46280737d0 2024-05-11T01:13:42.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.975+0000 7f46277fe640 1 --2- 172.21.15.52:0/4215164770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46280737d0 0x7f4628166b20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4620009d90 tx=0x7f462002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684a32fc9d8ba5b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:42.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.975+0000 7f46267fc640 1 -- 172.21.15.52:0/4215164770 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620007740 con 0x7f46280737d0 2024-05-11T01:13:42.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.975+0000 7f46267fc640 1 -- 172.21.15.52:0/4215164770 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46200078e0 con 0x7f46280737d0 2024-05-11T01:13:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.975+0000 7f46267fc640 1 -- 172.21.15.52:0/4215164770 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620007cb0 con 0x7f46280737d0 2024-05-11T01:13:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.976+0000 7f462d79d640 1 -- 172.21.15.52:0/4215164770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46280737d0 msgr2=0x7f4628166b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.976+0000 7f462d79d640 1 --2- 172.21.15.52:0/4215164770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46280737d0 0x7f4628166b20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4620009d90 tx=0x7f462002a6f0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.976+0000 7f462d79d640 1 -- 172.21.15.52:0/4215164770 shutdown_connections 2024-05-11T01:13:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.976+0000 7f462d79d640 1 --2- 172.21.15.52:0/4215164770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46280737d0 0x7f4628166b20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.976+0000 7f462d79d640 1 -- 172.21.15.52:0/4215164770 >> 172.21.15.52:0/4215164770 conn(0x7f46280b2a00 msgr2=0x7f46280b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.976+0000 7f462d79d640 1 -- 172.21.15.52:0/4215164770 shutdown_connections 2024-05-11T01:13:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.976+0000 7f462d79d640 1 -- 172.21.15.52:0/4215164770 wait complete. 2024-05-11T01:13:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.977+0000 7f462d79d640 1 Processor -- start 2024-05-11T01:13:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.977+0000 7f462d79d640 1 -- start start 2024-05-11T01:13:42.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f462d79d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4628072ce0 0x7f462814b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f462d79d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46280737d0 0x7f4628148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f462d79d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46280afb30 0x7f462815dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f462d79d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46280af610 con 0x7f4628072ce0 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f462d79d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46280b0340 con 0x7f46280737d0 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f462d79d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f462815e540 con 0x7f46280afb30 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f4626ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4628072ce0 0x7f462814b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f4626ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4628072ce0 0x7f462814b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36210/0 (socket says 172.21.15.52:36210) 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f4626ffd640 1 -- 172.21.15.52:0/1877302648 learned_addr learned my addr 172.21.15.52:0/1877302648 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f461e7fc640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46280737d0 0x7f4628148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.978+0000 7f46277fe640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46280afb30 0x7f462815dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f4626ffd640 1 -- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46280afb30 msgr2=0x7f462815dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f4626ffd640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46280afb30 0x7f462815dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f4626ffd640 1 -- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46280737d0 msgr2=0x7f4628148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f4626ffd640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46280737d0 0x7f4628148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f4626ffd640 1 -- 172.21.15.52:0/1877302648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462815e820 con 0x7f4628072ce0 2024-05-11T01:13:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f4626ffd640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4628072ce0 0x7f462814b700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f461800c500 tx=0x7f461800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f461ffff640 1 -- 172.21.15.52:0/1877302648 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618017070 con 0x7f4628072ce0 2024-05-11T01:13:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f461ffff640 1 -- 172.21.15.52:0/1877302648 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4618005ce0 con 0x7f4628072ce0 2024-05-11T01:13:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462815eb00 con 0x7f4628072ce0 2024-05-11T01:13:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f461ffff640 1 -- 172.21.15.52:0/1877302648 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180072f0 con 0x7f4628072ce0 2024-05-11T01:13:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.979+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462815e110 con 0x7f4628072ce0 2024-05-11T01:13:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.981+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4628069780 con 0x7f4628072ce0 2024-05-11T01:13:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.981+0000 7f461ffff640 1 -- 172.21.15.52:0/1877302648 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 98372+0+0 (secure 0 0 0) 0x7f461801a070 con 0x7f4628072ce0 2024-05-11T01:13:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.981+0000 7f461ffff640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4608076480 0x7f4608078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.982+0000 7f461e7fc640 1 -- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4608076480 msgr2=0x7f4608078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.982+0000 7f461ffff640 1 -- 172.21.15.52:0/1877302648 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6663+0+0 (secure 0 0 0) 0x7f4618096ea0 con 0x7f4628072ce0 2024-05-11T01:13:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.982+0000 7f461e7fc640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4608076480 0x7f4608078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:13:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:42.986+0000 7f461ffff640 1 -- 172.21.15.52:0/1877302648 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f461805fcb0 con 0x7f4628072ce0 2024-05-11T01:13:43.094 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:43.095+0000 7fca2c5ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T01:13:43.167 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:43.168+0000 7fbc6cbbf040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T01:13:43.178 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:43.180+0000 7fca2c5ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T01:13:43.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.183+0000 7f461e7fc640 1 -- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4608076480 msgr2=0x7f4608078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:43.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.183+0000 7f461e7fc640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4608076480 0x7f4608078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:13:43.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.218+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4628076850 con 0x7f4628072ce0 2024-05-11T01:13:43.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.218+0000 7f461ffff640 1 -- 172.21.15.52:0/1877302648 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7f4618063fb0 con 0x7f4628072ce0 2024-05-11T01:13:43.219 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:43.219 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3122737368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3122737368},{"type":"v1","addr":"172.21.15.119:6836","nonce":3122737368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/211602681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":211602681},{"type":"v1","addr":"172.21.15.119:6837","nonce":211602681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2079689543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2079689543},{"type":"v1","addr":"172.21.15.52:6835","nonce":2079689543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:44.660501+0000","modified":"2024-05-11T01:01:47.669355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4854":{"gid":4854,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1325827549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1325827549},{"type":"v1","addr":"172.21.15.52:6834","nonce":1325827549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4854,"qdb_cluster":[4854]},"id":2}]} 2024-05-11T01:13:43.219 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-05-11T01:13:43.220 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:43.221+0000 7fca2c5ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T01:13:43.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.220+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4608076480 msgr2=0x7f4608078940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:13:43.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.220+0000 7f462d79d640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4608076480 0x7f4608078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.220+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4628072ce0 msgr2=0x7f462814b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.221+0000 7f462d79d640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4628072ce0 0x7f462814b700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f461800c500 tx=0x7f461800c9c0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.221+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 shutdown_connections 2024-05-11T01:13:43.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.221+0000 7f462d79d640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46280afb30 0x7f462815dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.221+0000 7f462d79d640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46280737d0 0x7f4628148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.221+0000 7f462d79d640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f4608076480 0x7f4608078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.221+0000 7f462d79d640 1 --2- 172.21.15.52:0/1877302648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4628072ce0 0x7f462814b700 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.221+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 >> 172.21.15.52:0/1877302648 conn(0x7f46280b2a00 msgr2=0x7f4628158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:43.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.221+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 shutdown_connections 2024-05-11T01:13:43.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.221+0000 7f462d79d640 1 -- 172.21.15.52:0/1877302648 wait complete. 2024-05-11T01:13:43.233 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:13:43.259 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:43.260+0000 7fbc6cbbf040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T01:13:43.305 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:43.305+0000 7fbc6cbbf040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T01:13:43.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.337+0000 7f3273189640 1 Processor -- start 2024-05-11T01:13:43.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.337+0000 7f3273189640 1 -- start start 2024-05-11T01:13:43.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.337+0000 7f3273189640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f326c06c480 0x7f326c0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3273189640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f326c160bd0 0x7f326c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3273189640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f326c15d4a0 con 0x7f326c06c860 2024-05-11T01:13:43.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3273189640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f326c0cf6b0 con 0x7f326c06c480 2024-05-11T01:13:43.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3273189640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f326c0cece0 con 0x7f326c160bd0 2024-05-11T01:13:43.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3270f25640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f326c06c480 0x7f326c0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3270f25640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f326c06c480 0x7f326c0cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36222/0 (socket says 172.21.15.52:36222) 2024-05-11T01:13:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3270f25640 1 -- 172.21.15.52:0/2738506908 learned_addr learned my addr 172.21.15.52:0/2738506908 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3271726640 1 --2- 172.21.15.52:0/2738506908 >> v2:172.21.15.119:3300/0 conn(0x7f326c160bd0 0x7f326c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3270f25640 1 -- 172.21.15.52:0/2738506908 >> v2:172.21.15.119:3300/0 conn(0x7f326c160bd0 msgr2=0x7f326c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3270f25640 1 --2- 172.21.15.52:0/2738506908 >> v2:172.21.15.119:3300/0 conn(0x7f326c160bd0 0x7f326c161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3270f25640 1 -- 172.21.15.52:0/2738506908 >> v1:172.21.15.52:6789/0 conn(0x7f326c06c860 legacy=0x7f326c073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.338+0000 7f3270f25640 1 -- 172.21.15.52:0/2738506908 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c0d1680 con 0x7f326c06c480 2024-05-11T01:13:43.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.339+0000 7f3270f25640 1 --2- 172.21.15.52:0/2738506908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 0x7f326c0cf170 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f326c070600 tx=0x7f326002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d585a09f1d7dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.339+0000 7f326b7fe640 1 -- 172.21.15.52:0/2738506908 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260037070 con 0x7f326c06c480 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.339+0000 7f326b7fe640 1 -- 172.21.15.52:0/2738506908 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3260032440 con 0x7f326c06c480 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.339+0000 7f326b7fe640 1 -- 172.21.15.52:0/2738506908 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260032810 con 0x7f326c06c480 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.339+0000 7f3273189640 1 -- 172.21.15.52:0/2738506908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 msgr2=0x7f326c0cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.339+0000 7f3273189640 1 --2- 172.21.15.52:0/2738506908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 0x7f326c0cf170 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f326c070600 tx=0x7f326002a270 comp rx=0 tx=0).stop 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.340+0000 7f3273189640 1 -- 172.21.15.52:0/2738506908 shutdown_connections 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.340+0000 7f3273189640 1 --2- 172.21.15.52:0/2738506908 >> v2:172.21.15.119:3300/0 conn(0x7f326c160bd0 0x7f326c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.340+0000 7f3273189640 1 --2- 172.21.15.52:0/2738506908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 0x7f326c0cf170 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.340+0000 7f3273189640 1 -- 172.21.15.52:0/2738506908 >> 172.21.15.52:0/2738506908 conn(0x7f326c0b3a10 msgr2=0x7f326c0b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.340+0000 7f3273189640 1 -- 172.21.15.52:0/2738506908 shutdown_connections 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.340+0000 7f3273189640 1 -- 172.21.15.52:0/2738506908 wait complete. 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.341+0000 7f3273189640 1 Processor -- start 2024-05-11T01:13:43.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.341+0000 7f3273189640 1 -- start start 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3273189640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 0x7f326c0b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3273189640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f326c06c860 0x7f326c150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3273189640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f326c160bd0 0x7f326c15f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3273189640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f326c0d1330 con 0x7f326c06c480 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3273189640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f326c0ce970 con 0x7f326c06c860 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3273189640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f326c0ce070 con 0x7f326c160bd0 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3270f25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 0x7f326c0b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3270f25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 0x7f326c0b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36232/0 (socket says 172.21.15.52:36232) 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3270f25640 1 -- 172.21.15.52:0/4287689463 learned_addr learned my addr 172.21.15.52:0/4287689463 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f326bfff640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f326c06c860 0x7f326c150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3271726640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f326c160bd0 0x7f326c15f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3270f25640 1 -- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f326c160bd0 msgr2=0x7f326c15f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3270f25640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f326c160bd0 0x7f326c15f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3270f25640 1 -- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f326c06c860 msgr2=0x7f326c150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3270f25640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f326c06c860 0x7f326c150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3270f25640 1 -- 172.21.15.52:0/4287689463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c1602e0 con 0x7f326c06c480 2024-05-11T01:13:43.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f3270f25640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 0x7f326c0b56f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f326002a2b0 tx=0x7f32600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:43.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.342+0000 7f32697fa640 1 -- 172.21.15.52:0/4287689463 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260041020 con 0x7f326c06c480 2024-05-11T01:13:43.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.343+0000 7f32697fa640 1 -- 172.21.15.52:0/4287689463 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32600306c0 con 0x7f326c06c480 2024-05-11T01:13:43.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.343+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326c0b4c10 con 0x7f326c06c480 2024-05-11T01:13:43.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.343+0000 7f32697fa640 1 -- 172.21.15.52:0/4287689463 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260037040 con 0x7f326c06c480 2024-05-11T01:13:43.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.343+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326c157b40 con 0x7f326c06c480 2024-05-11T01:13:43.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.344+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f326c073fb0 con 0x7f326c06c480 2024-05-11T01:13:43.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.344+0000 7f32697fa640 1 -- 172.21.15.52:0/4287689463 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 98372+0+0 (secure 0 0 0) 0x7f3260044020 con 0x7f326c06c480 2024-05-11T01:13:43.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.345+0000 7f32697fa640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3250076480 0x7f3250078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.345+0000 7f326bfff640 1 -- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3250076480 msgr2=0x7f3250078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:43.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.345+0000 7f32697fa640 1 -- 172.21.15.52:0/4287689463 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6663+0+0 (secure 0 0 0) 0x7f32600b82b0 con 0x7f326c06c480 2024-05-11T01:13:43.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.345+0000 7f326bfff640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3250076480 0x7f3250078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:13:43.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.349+0000 7f32697fa640 1 -- 172.21.15.52:0/4287689463 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f326c073fb0 con 0x7f326c06c480 2024-05-11T01:13:43.382 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:43.383+0000 7fca2c5ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T01:13:43.428 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:43.429+0000 7fca2c5ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T01:13:43.478 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:43.479+0000 7fca2c5ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T01:13:43.543 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:43.483+0000 7fbc6cbbf040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T01:13:43.544 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:43.530+0000 7fbc6cbbf040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T01:13:43.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.546+0000 7f326bfff640 1 -- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3250076480 msgr2=0x7f3250078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:43.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.546+0000 7f326bfff640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3250076480 0x7f3250078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:13:43.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.580+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f326c069c00 con 0x7f326c06c480 2024-05-11T01:13:43.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.581+0000 7f32697fa640 1 -- 172.21.15.52:0/4287689463 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7f326c069c00 con 0x7f326c06c480 2024-05-11T01:13:43.581 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:43.581 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3122737368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3122737368},{"type":"v1","addr":"172.21.15.119:6836","nonce":3122737368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/211602681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":211602681},{"type":"v1","addr":"172.21.15.119:6837","nonce":211602681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2079689543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2079689543},{"type":"v1","addr":"172.21.15.52:6835","nonce":2079689543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:44.660501+0000","modified":"2024-05-11T01:01:47.669355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4854":{"gid":4854,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1325827549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1325827549},{"type":"v1","addr":"172.21.15.52:6834","nonce":1325827549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4854,"qdb_cluster":[4854]},"id":2}]} 2024-05-11T01:13:43.581 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-05-11T01:13:43.582 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:43.582+0000 7fbc6cbbf040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T01:13:43.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3250076480 msgr2=0x7f3250078940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3250076480 0x7f3250078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 msgr2=0x7f326c0b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 0x7f326c0b56f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f326002a2b0 tx=0x7f32600046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 shutdown_connections 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f326c160bd0 0x7f326c15f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f326c06c860 0x7f326c150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f3250076480 0x7f3250078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 --2- 172.21.15.52:0/4287689463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f326c06c480 0x7f326c0b56f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 >> 172.21.15.52:0/4287689463 conn(0x7f326c0b3a10 msgr2=0x7f326c0cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 shutdown_connections 2024-05-11T01:13:43.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.583+0000 7f3273189640 1 -- 172.21.15.52:0/4287689463 wait complete. 2024-05-11T01:13:43.589 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:43.590+0000 7fca2c5ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:13:43.595 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:13:43.702 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:43.702+0000 7fbc6cbbf040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T01:13:43.718 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:13:43.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:13:43.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.740+0000 7fd960589640 1 Processor -- start 2024-05-11T01:13:43.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.740+0000 7fd960589640 1 -- start start 2024-05-11T01:13:43.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd960589640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd95815f430 0x7fd95815fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd960589640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9581602b0 0x7fd9580d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd960589640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 0x7fd9580d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd960589640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd958163250 con 0x7fd9581602b0 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd960589640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd958163530 con 0x7fd95815f430 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd960589640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd958163810 con 0x7fd958160720 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95db24640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9581602b0 0x7fd9580d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95db24640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd9581602b0 0x7fd9580d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36244/0 (socket says 172.21.15.52:36244) 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95db24640 1 -- 172.21.15.52:0/1483671714 learned_addr learned my addr 172.21.15.52:0/1483671714 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95e325640 1 --2- 172.21.15.52:0/1483671714 >> v2:172.21.15.119:3300/0 conn(0x7fd95815f430 0x7fd95815fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95eb26640 1 --2- 172.21.15.52:0/1483671714 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 0x7fd9580d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95db24640 1 -- 172.21.15.52:0/1483671714 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 msgr2=0x7fd9580d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:43.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95db24640 1 --2- 172.21.15.52:0/1483671714 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 0x7fd9580d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95db24640 1 -- 172.21.15.52:0/1483671714 >> v2:172.21.15.119:3300/0 conn(0x7fd95815f430 msgr2=0x7fd95815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:43.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95db24640 1 --2- 172.21.15.52:0/1483671714 >> v2:172.21.15.119:3300/0 conn(0x7fd95815f430 0x7fd95815fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95db24640 1 -- 172.21.15.52:0/1483671714 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd958163af0 con 0x7fd9581602b0 2024-05-11T01:13:43.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.741+0000 7fd95e325640 1 --2- 172.21.15.52:0/1483671714 >> v2:172.21.15.119:3300/0 conn(0x7fd95815f430 0x7fd95815fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:43.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.742+0000 7fd95db24640 1 --2- 172.21.15.52:0/1483671714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 0x7fd9580d1b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd9580b6b80 tx=0x7fd94802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51fc2eebbf1bf21 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:43.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.742+0000 7fd95d323640 1 -- 172.21.15.52:0/1483671714 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948037070 con 0x7fd9581602b0 2024-05-11T01:13:43.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.742+0000 7fd95d323640 1 -- 172.21.15.52:0/1483671714 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd948032440 con 0x7fd9581602b0 2024-05-11T01:13:43.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.742+0000 7fd95d323640 1 -- 172.21.15.52:0/1483671714 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948032810 con 0x7fd9581602b0 2024-05-11T01:13:43.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.743+0000 7fd960589640 1 -- 172.21.15.52:0/1483671714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 msgr2=0x7fd9580d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.743+0000 7fd960589640 1 --2- 172.21.15.52:0/1483671714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 0x7fd9580d1b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd9580b6b80 tx=0x7fd94802a1e0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.743+0000 7fd960589640 1 -- 172.21.15.52:0/1483671714 shutdown_connections 2024-05-11T01:13:43.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.743+0000 7fd960589640 1 --2- 172.21.15.52:0/1483671714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 0x7fd9580d1b50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.743+0000 7fd960589640 1 --2- 172.21.15.52:0/1483671714 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 0x7fd9580d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.743+0000 7fd960589640 1 --2- 172.21.15.52:0/1483671714 >> v2:172.21.15.119:3300/0 conn(0x7fd95815f430 0x7fd95815fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.743+0000 7fd960589640 1 -- 172.21.15.52:0/1483671714 >> 172.21.15.52:0/1483671714 conn(0x7fd95808eb70 msgr2=0x7fd95809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:43.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.743+0000 7fd960589640 1 -- 172.21.15.52:0/1483671714 shutdown_connections 2024-05-11T01:13:43.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.743+0000 7fd960589640 1 -- 172.21.15.52:0/1483671714 wait complete. 2024-05-11T01:13:43.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.744+0000 7fd960589640 1 Processor -- start 2024-05-11T01:13:43.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd960589640 1 -- start start 2024-05-11T01:13:43.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd960589640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95815f430 0x7fd95815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd960589640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 0x7fd95815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd960589640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 0x7fd958161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd960589640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd958163030 con 0x7fd9581602b0 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd960589640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd958162eb0 con 0x7fd95815f430 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd960589640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd9580d25d0 con 0x7fd958160720 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd95db24640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 0x7fd95815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd95db24640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 0x7fd95815b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36256/0 (socket says 172.21.15.52:36256) 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.745+0000 7fd95db24640 1 -- 172.21.15.52:0/2646311840 learned_addr learned my addr 172.21.15.52:0/2646311840 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95e325640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95815f430 0x7fd95815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95eb26640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 0x7fd958161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95db24640 1 -- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 msgr2=0x7fd958161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95db24640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 0x7fd958161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95db24640 1 -- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95815f430 msgr2=0x7fd95815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95db24640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95815f430 0x7fd95815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95db24640 1 -- 172.21.15.52:0/2646311840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95815a9b0 con 0x7fd9581602b0 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95eb26640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 0x7fd958161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:43.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95e325640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95815f430 0x7fd95815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:43.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd95db24640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 0x7fd95815b9a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd94802a220 tx=0x7fd9480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:43.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd94effd640 1 -- 172.21.15.52:0/2646311840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948041020 con 0x7fd9581602b0 2024-05-11T01:13:43.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd94effd640 1 -- 172.21.15.52:0/2646311840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94803d620 con 0x7fd9581602b0 2024-05-11T01:13:43.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95815ce40 con 0x7fd9581602b0 2024-05-11T01:13:43.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd94effd640 1 -- 172.21.15.52:0/2646311840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948037040 con 0x7fd9581602b0 2024-05-11T01:13:43.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.746+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9580ca7c0 con 0x7fd9581602b0 2024-05-11T01:13:43.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.747+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd958069780 con 0x7fd9581602b0 2024-05-11T01:13:43.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.748+0000 7fd94effd640 1 -- 172.21.15.52:0/2646311840 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 98372+0+0 (secure 0 0 0) 0x7fd948044020 con 0x7fd9581602b0 2024-05-11T01:13:43.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.749+0000 7fd94effd640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd93c076480 0x7fd93c078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:43.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.749+0000 7fd94effd640 1 -- 172.21.15.52:0/2646311840 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6663+0+0 (secure 0 0 0) 0x7fd9480b8640 con 0x7fd9581602b0 2024-05-11T01:13:43.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.749+0000 7fd95e325640 1 -- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd93c076480 msgr2=0x7fd93c078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:43.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.749+0000 7fd95e325640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd93c076480 0x7fd93c078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:13:43.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.752+0000 7fd94effd640 1 -- 172.21.15.52:0/2646311840 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd94804b030 con 0x7fd9581602b0 2024-05-11T01:13:43.911 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:43.912+0000 7fca2c5ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T01:13:43.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.946+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9580b5360 con 0x7fd9581602b0 2024-05-11T01:13:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.947+0000 7fd94effd640 1 -- 172.21.15.52:0/2646311840 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+18128 (secure 0 0 0) 0x7fd9480813f0 con 0x7fd9581602b0 2024-05-11T01:13:43.948 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:43.948 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":46,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:13:42.012053+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:01:42.710844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:43.612754+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":226,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1820176553":"2024-05-12T01:01:57.869350+0000","172.21.15.119:0/1584246087":"2024-05-12T01:01:57.869350+0000","172.21.15.119:6833/3083030853":"2024-05-12T01:01:57.869350+0000","172.21.15.119:6832/3083030853":"2024-05-12T01:01:57.869350+0000","172.21.15.119:0/1042890889":"2024-05-12T01:01:57.869350+0000","172.21.15.119:0/1530920757":"2024-05-12T01:01:57.869350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":3},{"begin":11,"length":1},{"begin":13,"length":3},{"begin":17,"length":3},{"begin":21,"length":1},{"begin":23,"length":5},{"begin":29,"length":3},{"begin":33,"length":1},{"begin":35,"length":3},{"begin":39,"length":3},{"begin":43,"length":1},{"begin":46,"length":3},{"begin":50,"length":3},{"begin":54,"length":2},{"begin":57,"length":3},{"begin":61,"length":3},{"begin":65,"length":1},{"begin":67,"length":3},{"begin":71,"length":3},{"begin":75,"length":1},{"begin":77,"length":3},{"begin":81,"length":3},{"begin":86,"length":1},{"begin":88,"length":3},{"begin":92,"length":4},{"begin":97,"length":1},{"begin":99,"length":3},{"begin":103,"length":6},{"begin":110,"length":3},{"begin":150,"length":77}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":17,"length":1},{"begin":29,"length":1},{"begin":39,"length":1},{"begin":50,"length":1},{"begin":61,"length":1},{"begin":71,"length":1},{"begin":81,"length":1},{"begin":92,"length":1},{"begin":103,"length":1},{"begin":217,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:13:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd93c076480 msgr2=0x7fd93c078940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:13:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd93c076480 0x7fd93c078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 msgr2=0x7fd95815b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 0x7fd95815b9a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd94802a220 tx=0x7fd9480046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 shutdown_connections 2024-05-11T01:13:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd958160720 0x7fd958161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fd93c076480 0x7fd93c078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9581602b0 0x7fd95815b9a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 --2- 172.21.15.52:0/2646311840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95815f430 0x7fd95815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 >> 172.21.15.52:0/2646311840 conn(0x7fd95808eb70 msgr2=0x7fd9580cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 shutdown_connections 2024-05-11T01:13:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:43.949+0000 7fd960589640 1 -- 172.21.15.52:0/2646311840 wait complete. 2024-05-11T01:13:43.961 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T01:13:43.961 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:13:44.038 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:44.040+0000 7fca2c5ee040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T01:13:44.056 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:44.057+0000 7fbc6cbbf040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T01:13:44.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.072+0000 7f5ca15e4640 1 Processor -- start 2024-05-11T01:13:44.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.072+0000 7f5ca15e4640 1 -- start start 2024-05-11T01:13:44.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.073+0000 7f5ca15e4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5c9c15f580 0x7f5c9c0cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.073+0000 7f5ca15e4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c0d36f0 0x7f5c9c0d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.073+0000 7f5ca15e4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c9c0d0460 con 0x7f5c9c0d1500 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.073+0000 7f5ca15e4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9c0d0740 con 0x7f5c9c15f580 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.073+0000 7f5ca15e4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c0d0a20 con 0x7f5c9c0d36f0 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.073+0000 7f5c9affd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5c9c15f580 0x7f5c9c0cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.073+0000 7f5c9affd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5c9c15f580 0x7f5c9c0cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36266/0 (socket says 172.21.15.52:36266) 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.073+0000 7f5c9affd640 1 -- 172.21.15.52:0/3751790393 learned_addr learned my addr 172.21.15.52:0/3751790393 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.073+0000 7f5c9b7fe640 1 --2- 172.21.15.52:0/3751790393 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c0d36f0 0x7f5c9c0d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.074+0000 7f5c9affd640 1 -- 172.21.15.52:0/3751790393 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c0d36f0 msgr2=0x7f5c9c0d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.074+0000 7f5c9affd640 1 --2- 172.21.15.52:0/3751790393 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c0d36f0 0x7f5c9c0d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.074+0000 7f5c9affd640 1 -- 172.21.15.52:0/3751790393 >> v1:172.21.15.119:6789/0 conn(0x7f5c9c0d1500 legacy=0x7f5c9c0d56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.074+0000 7f5c9affd640 1 -- 172.21.15.52:0/3751790393 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c0d0d00 con 0x7f5c9c15f580 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.074+0000 7f5c9affd640 1 --2- 172.21.15.52:0/3751790393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c15f580 0x7f5c9c0cfd80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009870 tx=0x7f5c8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3330bdce3d384100 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:44.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.074+0000 7f5c99ffb640 1 -- 172.21.15.52:0/3751790393 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c037070 con 0x7f5c9c15f580 2024-05-11T01:13:44.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.075+0000 7f5c99ffb640 1 -- 172.21.15.52:0/3751790393 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c032440 con 0x7f5c9c15f580 2024-05-11T01:13:44.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.075+0000 7f5c99ffb640 1 -- 172.21.15.52:0/3751790393 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c032810 con 0x7f5c9c15f580 2024-05-11T01:13:44.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.075+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3751790393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c15f580 msgr2=0x7f5c9c0cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.075+0000 7f5ca15e4640 1 --2- 172.21.15.52:0/3751790393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c15f580 0x7f5c9c0cfd80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009870 tx=0x7f5c8c02a270 comp rx=0 tx=0).stop 2024-05-11T01:13:44.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.075+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3751790393 shutdown_connections 2024-05-11T01:13:44.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.075+0000 7f5ca15e4640 1 --2- 172.21.15.52:0/3751790393 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c0d36f0 0x7f5c9c0d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.075+0000 7f5ca15e4640 1 --2- 172.21.15.52:0/3751790393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c15f580 0x7f5c9c0cfd80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.075+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3751790393 >> 172.21.15.52:0/3751790393 conn(0x7f5c9c08eb80 msgr2=0x7f5c9c09e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:44.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.075+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3751790393 shutdown_connections 2024-05-11T01:13:44.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.076+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3751790393 wait complete. 2024-05-11T01:13:44.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.078+0000 7f5ca15e4640 1 Processor -- start 2024-05-11T01:13:44.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.078+0000 7f5ca15e4640 1 -- start start 2024-05-11T01:13:44.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5ca15e4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c0d1500 0x7f5c9c1f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5ca15e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9c0d36f0 0x7f5c9c1eaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5ca15e4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c15f580 0x7f5c9c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5ca15e4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c0c8cc0 con 0x7f5c9c0d1500 2024-05-11T01:13:44.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5ca15e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c15f960 con 0x7f5c9c0d36f0 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5ca15e4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c0c8e40 con 0x7f5c9c15f580 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c0d1500 0x7f5c9c1f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c0d1500 0x7f5c9c1f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36280/0 (socket says 172.21.15.52:36280) 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9affd640 1 -- 172.21.15.52:0/3114306204 learned_addr learned my addr 172.21.15.52:0/3114306204 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9b7fe640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c15f580 0x7f5c9c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9a7fc640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9c0d36f0 0x7f5c9c1eaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9b7fe640 1 -- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9c0d36f0 msgr2=0x7f5c9c1eaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9b7fe640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9c0d36f0 0x7f5c9c1eaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9b7fe640 1 -- 172.21.15.52:0/3114306204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c0d1500 msgr2=0x7f5c9c1f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9b7fe640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c0d1500 0x7f5c9c1f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9b7fe640 1 -- 172.21.15.52:0/3114306204 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c0b3c60 con 0x7f5c9c15f580 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.079+0000 7f5c9affd640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c0d1500 0x7f5c9c1f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.080+0000 7f5c9b7fe640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c15f580 0x7f5c9c0b43f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400efc0 tx=0x7f5c9400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.080+0000 7f5c837fe640 1 -- 172.21.15.52:0/3114306204 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94018020 con 0x7f5c9c15f580 2024-05-11T01:13:44.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.080+0000 7f5c837fe640 1 -- 172.21.15.52:0/3114306204 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c94014480 con 0x7f5c9c15f580 2024-05-11T01:13:44.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.080+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c1f6090 con 0x7f5c9c15f580 2024-05-11T01:13:44.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.080+0000 7f5c837fe640 1 -- 172.21.15.52:0/3114306204 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94014870 con 0x7f5c9c15f580 2024-05-11T01:13:44.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.080+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c0b3ed0 con 0x7f5c9c15f580 2024-05-11T01:13:44.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.081+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c0b40d0 con 0x7f5c9c15f580 2024-05-11T01:13:44.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.082+0000 7f5c837fe640 1 -- 172.21.15.52:0/3114306204 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 13) ==== 98372+0+0 (secure 0 0 0) 0x7f5c94005d10 con 0x7f5c9c15f580 2024-05-11T01:13:44.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.082+0000 7f5c837fe640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c880764b0 0x7f5c88078970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.082+0000 7f5c837fe640 1 -- 172.21.15.52:0/3114306204 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(47..47 src has 1..47) ==== 6535+0+0 (secure 0 0 0) 0x7f5c94098d30 con 0x7f5c9c15f580 2024-05-11T01:13:44.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.082+0000 7f5c9affd640 1 -- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c880764b0 msgr2=0x7f5c88078970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:44.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.082+0000 7f5c9affd640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c880764b0 0x7f5c88078970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:13:44.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.090+0000 7f5c837fe640 1 -- 172.21.15.52:0/3114306204 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5c9c0b40d0 con 0x7f5c9c15f580 2024-05-11T01:13:44.191 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:44.192+0000 7fbc6cbbf040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T01:13:44.253 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:44.254+0000 7fca2c5ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T01:13:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.283+0000 7f5c9affd640 1 -- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c880764b0 msgr2=0x7f5c88078970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.283+0000 7f5c9affd640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c880764b0 0x7f5c88078970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:13:44.302 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:44.303+0000 7fca2c5ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T01:13:44.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.322+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c9c0cb4c0 con 0x7f5c9c15f580 2024-05-11T01:13:44.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.323+0000 7f5c837fe640 1 -- 172.21.15.52:0/3114306204 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7f5c94022030 con 0x7f5c9c15f580 2024-05-11T01:13:44.323 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:44.323 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3122737368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3122737368},{"type":"v1","addr":"172.21.15.119:6836","nonce":3122737368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/211602681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":211602681},{"type":"v1","addr":"172.21.15.119:6837","nonce":211602681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2079689543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2079689543},{"type":"v1","addr":"172.21.15.52:6835","nonce":2079689543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:44.660501+0000","modified":"2024-05-11T01:01:47.669355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4854":{"gid":4854,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1325827549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1325827549},{"type":"v1","addr":"172.21.15.52:6834","nonce":1325827549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4854,"qdb_cluster":[4854]},"id":2}]} 2024-05-11T01:13:44.323 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-05-11T01:13:44.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c880764b0 msgr2=0x7f5c88078970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:13:44.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c880764b0 0x7f5c88078970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c15f580 msgr2=0x7f5c9c0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c15f580 0x7f5c9c0b43f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400efc0 tx=0x7f5c9400c490 comp rx=0 tx=0).stop 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 shutdown_connections 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5c9c15f580 0x7f5c9c0b43f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9c0d36f0 0x7f5c9c1eaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f5c880764b0 0x7f5c88078970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 --2- 172.21.15.52:0/3114306204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c9c0d1500 0x7f5c9c1f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 >> 172.21.15.52:0/3114306204 conn(0x7f5c9c08eb80 msgr2=0x7f5c9c0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 shutdown_connections 2024-05-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.325+0000 7f5ca15e4640 1 -- 172.21.15.52:0/3114306204 wait complete. 2024-05-11T01:13:44.337 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:13:44.417 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:44.418+0000 7fca2c5ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T01:13:44.418 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:44.418+0000 7fbc6cbbf040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T01:13:44.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.440+0000 7feb366e1640 1 Processor -- start 2024-05-11T01:13:44.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.441+0000 7feb366e1640 1 -- start start 2024-05-11T01:13:44.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.441+0000 7feb366e1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb3009c130 0x7feb300d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.441+0000 7feb366e1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb300cd6e0 con 0x7feb300b55b0 2024-05-11T01:13:44.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.441+0000 7feb366e1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb300cd9c0 con 0x7feb300b8a10 2024-05-11T01:13:44.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.441+0000 7feb366e1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7feb300ce2f0 con 0x7feb3009c130 2024-05-11T01:13:44.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.442+0000 7feb34c7e640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7feb300b55b0 0x7feb300ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36344/0 (socket says 172.21.15.52:36344) 2024-05-11T01:13:44.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.442+0000 7feb34c7e640 1 -- 172.21.15.52:0/3156031021 learned_addr learned my addr 172.21.15.52:0/3156031021 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:44.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.442+0000 7feb2f7fe640 1 --2- 172.21.15.52:0/3156031021 >> v2:172.21.15.119:3300/0 conn(0x7feb3009c130 0x7feb300d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.442+0000 7feb2effd640 1 -- 172.21.15.52:0/3156031021 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3493648109 0 0) 0x7feb300cd6e0 con 0x7feb300b55b0 2024-05-11T01:13:44.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.442+0000 7feb2effd640 1 -- 172.21.15.52:0/3156031021 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb1c003520 con 0x7feb300b55b0 2024-05-11T01:13:44.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.442+0000 7feb2f7fe640 1 -- 172.21.15.52:0/3156031021 >> v1:172.21.15.119:6789/0 conn(0x7feb300b8a10 legacy=0x7feb3009ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.442+0000 7feb2effd640 1 -- 172.21.15.52:0/3156031021 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234318698 0 0) 0x7feb300cd9c0 con 0x7feb300b8a10 2024-05-11T01:13:44.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.442+0000 7feb2f7fe640 1 -- 172.21.15.52:0/3156031021 >> v1:172.21.15.52:6789/0 conn(0x7feb300b55b0 legacy=0x7feb300ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.442+0000 7feb2f7fe640 1 -- 172.21.15.52:0/3156031021 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb300cdca0 con 0x7feb3009c130 2024-05-11T01:13:44.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.443+0000 7feb2f7fe640 1 --2- 172.21.15.52:0/3156031021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb3009c130 0x7feb300d1cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb180097a0 tx=0x7feb1802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78c3b6530f0100 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:44.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.443+0000 7feb2effd640 1 -- 172.21.15.52:0/3156031021 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18037070 con 0x7feb3009c130 2024-05-11T01:13:44.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.443+0000 7feb2effd640 1 -- 172.21.15.52:0/3156031021 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb18032440 con 0x7feb3009c130 2024-05-11T01:13:44.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.443+0000 7feb2effd640 1 -- 172.21.15.52:0/3156031021 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18032810 con 0x7feb3009c130 2024-05-11T01:13:44.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.444+0000 7feb366e1640 1 -- 172.21.15.52:0/3156031021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb3009c130 msgr2=0x7feb300d1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.444+0000 7feb366e1640 1 --2- 172.21.15.52:0/3156031021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb3009c130 0x7feb300d1cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb180097a0 tx=0x7feb1802a1a0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.444+0000 7feb366e1640 1 -- 172.21.15.52:0/3156031021 shutdown_connections 2024-05-11T01:13:44.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.444+0000 7feb366e1640 1 --2- 172.21.15.52:0/3156031021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb3009c130 0x7feb300d1cf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.444+0000 7feb366e1640 1 -- 172.21.15.52:0/3156031021 >> 172.21.15.52:0/3156031021 conn(0x7feb300b4a90 msgr2=0x7feb300b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:44.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.444+0000 7feb366e1640 1 -- 172.21.15.52:0/3156031021 shutdown_connections 2024-05-11T01:13:44.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.444+0000 7feb366e1640 1 -- 172.21.15.52:0/3156031021 wait complete. 2024-05-11T01:13:44.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.445+0000 7feb366e1640 1 Processor -- start 2024-05-11T01:13:44.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.445+0000 7feb366e1640 1 -- start start 2024-05-11T01:13:44.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb366e1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3009c130 0x7feb3014c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb366e1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb300b55b0 0x7feb30160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb366e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb300b8a10 0x7feb300cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb366e1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb300b5db0 con 0x7feb3009c130 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb366e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb300b8df0 con 0x7feb300b8a10 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb366e1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7feb3006ee60 con 0x7feb300b55b0 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb2ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3009c130 0x7feb3014c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb2ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3009c130 0x7feb3014c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36292/0 (socket says 172.21.15.52:36292) 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb2ffff640 1 -- 172.21.15.52:0/1936493747 learned_addr learned my addr 172.21.15.52:0/1936493747 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb34c7e640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb300b8a10 0x7feb300cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb2f7fe640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb300b55b0 0x7feb30160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb2ffff640 1 -- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb300b55b0 msgr2=0x7feb30160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb2ffff640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb300b55b0 0x7feb30160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb2ffff640 1 -- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb300b8a10 msgr2=0x7feb300cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb2ffff640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb300b8a10 0x7feb300cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.446+0000 7feb2ffff640 1 -- 172.21.15.52:0/1936493747 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3014d080 con 0x7feb3009c130 2024-05-11T01:13:44.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.447+0000 7feb2ffff640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3009c130 0x7feb3014c8e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c500 tx=0x7feb2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:44.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.447+0000 7feb2cff9640 1 -- 172.21.15.52:0/1936493747 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20017070 con 0x7feb3009c130 2024-05-11T01:13:44.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.447+0000 7feb2cff9640 1 -- 172.21.15.52:0/1936493747 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb200035f0 con 0x7feb3009c130 2024-05-11T01:13:44.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.447+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb301613a0 con 0x7feb3009c130 2024-05-11T01:13:44.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.447+0000 7feb2cff9640 1 -- 172.21.15.52:0/1936493747 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20003a20 con 0x7feb3009c130 2024-05-11T01:13:44.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.447+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb30162bd0 con 0x7feb3009c130 2024-05-11T01:13:44.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.448+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb30069780 con 0x7feb3009c130 2024-05-11T01:13:44.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.449+0000 7feb2cff9640 1 -- 172.21.15.52:0/1936493747 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 98372+0+0 (secure 0 0 0) 0x7feb2001a070 con 0x7feb3009c130 2024-05-11T01:13:44.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.449+0000 7feb2cff9640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7feb10076480 0x7feb10078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.449+0000 7feb2f7fe640 1 -- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7feb10076480 msgr2=0x7feb10078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:44.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.449+0000 7feb2cff9640 1 -- 172.21.15.52:0/1936493747 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6535+0+0 (secure 0 0 0) 0x7feb20096f50 con 0x7feb3009c130 2024-05-11T01:13:44.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.449+0000 7feb2f7fe640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7feb10076480 0x7feb10078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:13:44.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.453+0000 7feb2cff9640 1 -- 172.21.15.52:0/1936493747 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feb2005fde0 con 0x7feb3009c130 2024-05-11T01:13:44.472 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:44.472+0000 7fbc6cbbf040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T01:13:44.593 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:44.593+0000 7fbc6cbbf040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T01:13:44.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.650+0000 7feb2f7fe640 1 -- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7feb10076480 msgr2=0x7feb10078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:44.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.650+0000 7feb2f7fe640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7feb10076480 0x7feb10078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:13:44.689 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:44.690+0000 7fca2c5ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T01:13:44.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.689+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb30072020 con 0x7feb3009c130 2024-05-11T01:13:44.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.690+0000 7feb2cff9640 1 -- 172.21.15.52:0/1936493747 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7feb20068420 con 0x7feb3009c130 2024-05-11T01:13:44.690 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:44.690 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3122737368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3122737368},{"type":"v1","addr":"172.21.15.119:6836","nonce":3122737368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/211602681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":211602681},{"type":"v1","addr":"172.21.15.119:6837","nonce":211602681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2079689543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2079689543},{"type":"v1","addr":"172.21.15.52:6835","nonce":2079689543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:01:44.660501+0000","modified":"2024-05-11T01:01:47.669355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4854":{"gid":4854,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1325827549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1325827549},{"type":"v1","addr":"172.21.15.52:6834","nonce":1325827549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4854,"qdb_cluster":[4854]},"id":2}]} 2024-05-11T01:13:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-05-11T01:13:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7feb10076480 msgr2=0x7feb10078940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:13:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7feb10076480 0x7feb10078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3009c130 msgr2=0x7feb3014c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3009c130 0x7feb3014c8e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c500 tx=0x7feb2000c9c0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 shutdown_connections 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb300b8a10 0x7feb300cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb300b55b0 0x7feb30160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7feb10076480 0x7feb10078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 --2- 172.21.15.52:0/1936493747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3009c130 0x7feb3014c8e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 >> 172.21.15.52:0/1936493747 conn(0x7feb300b4a90 msgr2=0x7feb3015f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 shutdown_connections 2024-05-11T01:13:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.692+0000 7feb366e1640 1 -- 172.21.15.52:0/1936493747 wait complete. 2024-05-11T01:13:44.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:13:44.734 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:44.735+0000 7fca2c5ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T01:13:44.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.812+0000 7feffef9e640 1 Processor -- start 2024-05-11T01:13:44.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.812+0000 7feffef9e640 1 -- start start 2024-05-11T01:13:44.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffef9e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 0x7feff80cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffef9e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feff80b2060 0x7feff8071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffef9e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff80cff90 con 0x7feff80cd820 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffef9e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7feff80d0270 con 0x7feff80b2060 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffef9e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7feff80d0550 con 0x7feff815cf70 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffe79d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feff80b2060 0x7feff8071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffe79d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feff80b2060 0x7feff8071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36302/0 (socket says 172.21.15.52:36302) 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffe79d640 1 -- 172.21.15.52:0/2297782358 learned_addr learned my addr 172.21.15.52:0/2297782358 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffdf9c640 1 --2- 172.21.15.52:0/2297782358 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 0x7feff80cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffe79d640 1 -- 172.21.15.52:0/2297782358 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 msgr2=0x7feff80cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffe79d640 1 --2- 172.21.15.52:0/2297782358 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 0x7feff80cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffe79d640 1 -- 172.21.15.52:0/2297782358 >> v1:172.21.15.119:6789/0 conn(0x7feff80cd820 legacy=0x7feff80d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.813+0000 7feffe79d640 1 -- 172.21.15.52:0/2297782358 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff8070a30 con 0x7feff80b2060 2024-05-11T01:13:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.814+0000 7feffe79d640 1 --2- 172.21.15.52:0/2297782358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80b2060 0x7feff8071230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7feff0009e50 tx=0x7feff002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b297f1f960c16301 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.814+0000 7feffcf9a640 1 -- 172.21.15.52:0/2297782358 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0036070 con 0x7feff80b2060 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.814+0000 7feffcf9a640 1 -- 172.21.15.52:0/2297782358 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff0031600 con 0x7feff80b2060 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.814+0000 7feffcf9a640 1 -- 172.21.15.52:0/2297782358 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00319d0 con 0x7feff80b2060 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.815+0000 7feffef9e640 1 -- 172.21.15.52:0/2297782358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80b2060 msgr2=0x7feff8071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.815+0000 7feffef9e640 1 --2- 172.21.15.52:0/2297782358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80b2060 0x7feff8071230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7feff0009e50 tx=0x7feff002a710 comp rx=0 tx=0).stop 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.815+0000 7feffef9e640 1 -- 172.21.15.52:0/2297782358 shutdown_connections 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.815+0000 7feffef9e640 1 --2- 172.21.15.52:0/2297782358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80b2060 0x7feff8071230 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.815+0000 7feffef9e640 1 --2- 172.21.15.52:0/2297782358 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 0x7feff80cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.815+0000 7feffef9e640 1 -- 172.21.15.52:0/2297782358 >> 172.21.15.52:0/2297782358 conn(0x7feff809ba20 msgr2=0x7feff809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.815+0000 7feffef9e640 1 -- 172.21.15.52:0/2297782358 shutdown_connections 2024-05-11T01:13:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.815+0000 7feffef9e640 1 -- 172.21.15.52:0/2297782358 wait complete. 2024-05-11T01:13:44.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.816+0000 7feffef9e640 1 Processor -- start 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.816+0000 7feffef9e640 1 -- start start 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffef9e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80b2060 0x7feff8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffef9e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80cd820 0x7feff80b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffef9e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 0x7feff80b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffef9e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feff80cfbf0 con 0x7feff80cd820 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffef9e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feff815d350 con 0x7feff80b2060 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffef9e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7feff80cfd70 con 0x7feff815cf70 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffd79b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80cd820 0x7feff80b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffd79b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80cd820 0x7feff80b7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36314/0 (socket says 172.21.15.52:36314) 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffd79b640 1 -- 172.21.15.52:0/799247630 learned_addr learned my addr 172.21.15.52:0/799247630 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffdf9c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80b2060 0x7feff8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.817+0000 7feffe79d640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 0x7feff80b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7feffdf9c640 1 -- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 msgr2=0x7feff80b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7feffdf9c640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 0x7feff80b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7feffdf9c640 1 -- 172.21.15.52:0/799247630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80cd820 msgr2=0x7feff80b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7feffdf9c640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80cd820 0x7feff80b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7feffdf9c640 1 -- 172.21.15.52:0/799247630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff8156180 con 0x7feff80b2060 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7feffdf9c640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80b2060 0x7feff8155c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feff400e8a0 tx=0x7feff400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7fefee7fc640 1 -- 172.21.15.52:0/799247630 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4017070 con 0x7feff80b2060 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7fefee7fc640 1 -- 172.21.15.52:0/799247630 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff400c940 con 0x7feff80b2060 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff80b51f0 con 0x7feff80b2060 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7fefee7fc640 1 -- 172.21.15.52:0/799247630 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4009e60 con 0x7feff80b2060 2024-05-11T01:13:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.818+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff80b31a0 con 0x7feff80b2060 2024-05-11T01:13:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.819+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff8069c00 con 0x7feff80b2060 2024-05-11T01:13:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.820+0000 7fefee7fc640 1 -- 172.21.15.52:0/799247630 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) ==== 98372+0+0 (secure 0 0 0) 0x7feff401a070 con 0x7feff80b2060 2024-05-11T01:13:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.820+0000 7fefee7fc640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefdc076480 0x7fefdc078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.820+0000 7feffd79b640 1 -- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefdc076480 msgr2=0x7fefdc078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.820+0000 7fefee7fc640 1 -- 172.21.15.52:0/799247630 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6535+0+0 (secure 0 0 0) 0x7feff4097d30 con 0x7feff80b2060 2024-05-11T01:13:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.821+0000 7feffd79b640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefdc076480 0x7fefdc078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:13:44.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:44.825+0000 7fefee7fc640 1 -- 172.21.15.52:0/799247630 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feff4060b20 con 0x7feff80b2060 2024-05-11T01:13:44.881 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:44.882+0000 7fca2c5ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T01:13:44.894 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:44.894+0000 7fbc6cbbf040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T01:13:44.941 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:44.941+0000 7fbc6cbbf040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T01:13:45.015 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.016+0000 7fca2c5ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T01:13:45.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.022+0000 7feffd79b640 1 -- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefdc076480 msgr2=0x7fefdc078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:45.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.022+0000 7feffd79b640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefdc076480 0x7fefdc078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:13:45.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.025+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feff80b5710 con 0x7feff80b2060 2024-05-11T01:13:45.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.026+0000 7fefee7fc640 1 -- 172.21.15.52:0/799247630 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+18733 (secure 0 0 0) 0x7feff4064e20 con 0x7feff80b2060 2024-05-11T01:13:45.030 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:45.030 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":48,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:13:44.997484+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:01:42.710844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-11T01:01:43.612754+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":227,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1820176553":"2024-05-12T01:01:57.869350+0000","172.21.15.119:0/1584246087":"2024-05-12T01:01:57.869350+0000","172.21.15.119:6833/3083030853":"2024-05-12T01:01:57.869350+0000","172.21.15.119:6832/3083030853":"2024-05-12T01:01:57.869350+0000","172.21.15.119:0/1042890889":"2024-05-12T01:01:57.869350+0000","172.21.15.119:0/1530920757":"2024-05-12T01:01:57.869350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":9,"length":1},{"begin":13,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":46,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":57,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":67,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":77,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":94,"length":2},{"begin":99,"length":1},{"begin":103,"length":1},{"begin":106,"length":1},{"begin":110,"length":1},{"begin":112,"length":1},{"begin":114,"length":1},{"begin":194,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":8,"length":1},{"begin":11,"length":1},{"begin":14,"length":2},{"begin":18,"length":1},{"begin":21,"length":1},{"begin":24,"length":1},{"begin":26,"length":2},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":43,"length":1},{"begin":47,"length":2},{"begin":51,"length":1},{"begin":54,"length":2},{"begin":58,"length":2},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":68,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":2},{"begin":82,"length":1},{"begin":86,"length":1},{"begin":89,"length":2},{"begin":93,"length":1},{"begin":97,"length":1},{"begin":100,"length":2},{"begin":104,"length":2},{"begin":107,"length":2},{"begin":111,"length":1},{"begin":150,"length":44}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:13:45.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.028+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefdc076480 msgr2=0x7fefdc078940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:13:45.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefdc076480 0x7fefdc078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80b2060 msgr2=0x7feff8155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80b2060 0x7feff8155c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feff400e8a0 tx=0x7feff400ed60 comp rx=0 tx=0).stop 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 shutdown_connections 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feff815cf70 0x7feff80b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7fefdc076480 0x7fefdc078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feff80cd820 0x7feff80b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 --2- 172.21.15.52:0/799247630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80b2060 0x7feff8155c40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 >> 172.21.15.52:0/799247630 conn(0x7feff809ba20 msgr2=0x7feff80b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 shutdown_connections 2024-05-11T01:13:45.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.029+0000 7feffef9e640 1 -- 172.21.15.52:0/799247630 wait complete. 2024-05-11T01:13:45.042 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T01:13:45.066 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.067+0000 7fca2c5ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T01:13:45.100 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:45.100+0000 7fbc6cbbf040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T01:13:45.124 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.125+0000 7fca2c5ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T01:13:45.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.150+0000 7f79bf475640 1 Processor -- start 2024-05-11T01:13:45.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.150+0000 7f79bf475640 1 -- start start 2024-05-11T01:13:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bf475640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79b815f580 0x7f79b80cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bf475640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b80d1500 0x7f79b80d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bf475640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b80d5be0 con 0x7f79b80d36f0 2024-05-11T01:13:45.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bf475640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f79b80d0460 con 0x7f79b815f580 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bf475640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f79b80d0740 con 0x7f79b80d1500 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bda12640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f79b80d36f0 0x7f79b80d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36350/0 (socket says 172.21.15.52:36350) 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bda12640 1 -- 172.21.15.52:0/488443146 learned_addr learned my addr 172.21.15.52:0/488443146 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bca10640 1 --2- 172.21.15.52:0/488443146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b80d1500 0x7f79b80d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bd211640 1 --2- 172.21.15.52:0/488443146 >> v2:172.21.15.119:3300/0 conn(0x7f79b815f580 0x7f79b80cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79affff640 1 -- 172.21.15.52:0/488443146 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4081769196 0 0) 0x7f79b80d5be0 con 0x7f79b80d36f0 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bca10640 1 -- 172.21.15.52:0/488443146 >> v2:172.21.15.119:3300/0 conn(0x7f79b815f580 msgr2=0x7f79b80cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bca10640 1 --2- 172.21.15.52:0/488443146 >> v2:172.21.15.119:3300/0 conn(0x7f79b815f580 0x7f79b80cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bca10640 1 -- 172.21.15.52:0/488443146 >> v1:172.21.15.52:6789/0 conn(0x7f79b80d36f0 legacy=0x7f79b80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.151+0000 7f79bca10640 1 -- 172.21.15.52:0/488443146 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b80d0a20 con 0x7f79b80d1500 2024-05-11T01:13:45.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.152+0000 7f79bca10640 1 --2- 172.21.15.52:0/488443146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b80d1500 0x7f79b80d56a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79a80097a0 tx=0x7f79a802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b7dd0a2942485b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.152+0000 7f79affff640 1 -- 172.21.15.52:0/488443146 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8037070 con 0x7f79b80d1500 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.152+0000 7f79affff640 1 -- 172.21.15.52:0/488443146 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a8032440 con 0x7f79b80d1500 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.152+0000 7f79affff640 1 -- 172.21.15.52:0/488443146 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8032810 con 0x7f79b80d1500 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.153+0000 7f79bf475640 1 -- 172.21.15.52:0/488443146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b80d1500 msgr2=0x7f79b80d56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.153+0000 7f79bf475640 1 --2- 172.21.15.52:0/488443146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b80d1500 0x7f79b80d56a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79a80097a0 tx=0x7f79a802a180 comp rx=0 tx=0).stop 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.153+0000 7f79bf475640 1 -- 172.21.15.52:0/488443146 shutdown_connections 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.153+0000 7f79bf475640 1 --2- 172.21.15.52:0/488443146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b80d1500 0x7f79b80d56a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.153+0000 7f79bf475640 1 --2- 172.21.15.52:0/488443146 >> v2:172.21.15.119:3300/0 conn(0x7f79b815f580 0x7f79b80cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.153+0000 7f79bf475640 1 -- 172.21.15.52:0/488443146 >> 172.21.15.52:0/488443146 conn(0x7f79b808eb80 msgr2=0x7f79b809e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.153+0000 7f79bf475640 1 -- 172.21.15.52:0/488443146 shutdown_connections 2024-05-11T01:13:45.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.153+0000 7f79bf475640 1 -- 172.21.15.52:0/488443146 wait complete. 2024-05-11T01:13:45.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.154+0000 7f79bf475640 1 Processor -- start 2024-05-11T01:13:45.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.154+0000 7f79bf475640 1 -- start start 2024-05-11T01:13:45.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bf475640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b80d1500 0x7f79b8158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bf475640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79b80d36f0 0x7f79b80cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bf475640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b815f580 0x7f79b815c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bf475640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79b815f960 con 0x7f79b80d1500 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bf475640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f79b80d0fe0 con 0x7f79b80d36f0 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bf475640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f79b80d3ba0 con 0x7f79b815f580 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bd211640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b80d1500 0x7f79b8158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bd211640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b80d1500 0x7f79b8158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36328/0 (socket says 172.21.15.52:36328) 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bd211640 1 -- 172.21.15.52:0/2424103480 learned_addr learned my addr 172.21.15.52:0/2424103480 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bca10640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79b80d36f0 0x7f79b80cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.155+0000 7f79bda12640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b815f580 0x7f79b815c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79bd211640 1 -- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b815f580 msgr2=0x7f79b815c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79bd211640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b815f580 0x7f79b815c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79bd211640 1 -- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79b80d36f0 msgr2=0x7f79b80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79bd211640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79b80d36f0 0x7f79b80cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79bd211640 1 -- 172.21.15.52:0/2424103480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b80cf510 con 0x7f79b80d1500 2024-05-11T01:13:45.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79bd211640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b80d1500 0x7f79b8158b00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f79b400e8a0 tx=0x7f79b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:45.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79adffb640 1 -- 172.21.15.52:0/2424103480 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4017070 con 0x7f79b80d1500 2024-05-11T01:13:45.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79adffb640 1 -- 172.21.15.52:0/2424103480 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b400c940 con 0x7f79b80d1500 2024-05-11T01:13:45.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b80b7ec0 con 0x7f79b80d1500 2024-05-11T01:13:45.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79adffb640 1 -- 172.21.15.52:0/2424103480 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b400cd30 con 0x7f79b80d1500 2024-05-11T01:13:45.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.156+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b80b8440 con 0x7f79b80d1500 2024-05-11T01:13:45.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.157+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b8069780 con 0x7f79b80d1500 2024-05-11T01:13:45.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.158+0000 7f79adffb640 1 -- 172.21.15.52:0/2424103480 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 98372+0+0 (secure 0 0 0) 0x7f79b401a070 con 0x7f79b80d1500 2024-05-11T01:13:45.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.159+0000 7f79adffb640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f799c076480 0x7f799c078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.159+0000 7f79bca10640 1 -- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f799c076480 msgr2=0x7f799c078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:45.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.159+0000 7f79adffb640 1 -- 172.21.15.52:0/2424103480 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 7063+0+0 (secure 0 0 0) 0x7f79b4097b90 con 0x7f79b80d1500 2024-05-11T01:13:45.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.159+0000 7f79bca10640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f799c076480 0x7f799c078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:13:45.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.163+0000 7f79adffb640 1 -- 172.21.15.52:0/2424103480 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f79b4060770 con 0x7f79b80d1500 2024-05-11T01:13:45.168 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.169+0000 7fca2c5ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T01:13:45.247 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:45.247+0000 7fbc6cbbf040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T01:13:45.298 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:45.298+0000 7fbc6cbbf040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T01:13:45.357 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:45.357+0000 7fbc6cbbf040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T01:13:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.360+0000 7f79bca10640 1 -- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f799c076480 msgr2=0x7f799c078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.360+0000 7f79bca10640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f799c076480 0x7f799c078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T01:13:45.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.395+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f79b80cb0f0 con 0x7f79b80d1500 2024-05-11T01:13:45.395 INFO:tasks.ceph.mon.a.smithi052.stderr:2024-05-11T01:13:45.396+0000 7f9c6a575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T01:13:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.398+0000 7f79adffb640 1 -- 172.21.15.52:0/2424103480 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f79b4064a70 con 0x7f79b80d1500 2024-05-11T01:13:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f799c076480 msgr2=0x7f799c078940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f799c076480 0x7f799c078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b80d1500 msgr2=0x7f79b8158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b80d1500 0x7f79b8158b00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f79b400e8a0 tx=0x7f79b400ed60 comp rx=0 tx=0).stop 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 shutdown_connections 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f79b815f580 0x7f79b815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79b80d36f0 0x7f79b80cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f799c076480 0x7f799c078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 --2- 172.21.15.52:0/2424103480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b80d1500 0x7f79b8158b00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 >> 172.21.15.52:0/2424103480 conn(0x7f79b808eb80 msgr2=0x7f79b806e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 shutdown_connections 2024-05-11T01:13:45.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.402+0000 7f79bf475640 1 -- 172.21.15.52:0/2424103480 wait complete. 2024-05-11T01:13:45.405 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:45.405+0000 7fbc6cbbf040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T01:13:45.415 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T01:13:45.439 INFO:tasks.ceph.mds.a.smithi052.stdout:starting mds.a at 2024-05-11T01:13:45.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.523+0000 7f2ae8789640 1 Processor -- start 2024-05-11T01:13:45.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.523+0000 7f2ae8789640 1 -- start start 2024-05-11T01:13:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.524+0000 7f2ae8789640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2ae0162c00 0x7f2ae00b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.524+0000 7f2ae8789640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae00b67c0 con 0x7f2ae00d1cc0 2024-05-11T01:13:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.524+0000 7f2ae8789640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae00d09b0 con 0x7f2ae00d20a0 2024-05-11T01:13:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.524+0000 7f2ae8789640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae00d2d50 con 0x7f2ae0162c00 2024-05-11T01:13:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.524+0000 7f2ae6525640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f2ae00d1cc0 0x7f2ae00b60b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36370/0 (socket says 172.21.15.52:36370) 2024-05-11T01:13:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.524+0000 7f2ae6525640 1 -- 172.21.15.52:0/1225099100 learned_addr learned my addr 172.21.15.52:0/1225099100 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.525+0000 7f2ae6d26640 1 --2- 172.21.15.52:0/1225099100 >> v2:172.21.15.119:3300/0 conn(0x7f2ae0162c00 0x7f2ae00b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:45.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.525+0000 7f2ae5523640 1 -- 172.21.15.52:0/1225099100 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356097186 0 0) 0x7f2ae00b67c0 con 0x7f2ae00d1cc0 2024-05-11T01:13:45.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.525+0000 7f2ae5523640 1 -- 172.21.15.52:0/1225099100 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2acc003520 con 0x7f2ae00d1cc0 2024-05-11T01:13:45.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.525+0000 7f2ae5523640 1 -- 172.21.15.52:0/1225099100 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1235921160 0 0) 0x7f2ae00d09b0 con 0x7f2ae00d20a0 2024-05-11T01:13:45.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.525+0000 7f2ae5523640 1 -- 172.21.15.52:0/1225099100 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ae00b67c0 con 0x7f2ae00d20a0 2024-05-11T01:13:45.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.525+0000 7f2ae6d26640 1 -- 172.21.15.52:0/1225099100 >> v1:172.21.15.119:6789/0 conn(0x7f2ae00d20a0 legacy=0x7f2ae01624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.525+0000 7f2ae6d26640 1 -- 172.21.15.52:0/1225099100 >> v1:172.21.15.52:6789/0 conn(0x7f2ae00d1cc0 legacy=0x7f2ae00b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.525+0000 7f2ae6d26640 1 -- 172.21.15.52:0/1225099100 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae00d0c90 con 0x7f2ae0162c00 2024-05-11T01:13:45.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.526+0000 7f2ae6d26640 1 --2- 172.21.15.52:0/1225099100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae0162c00 0x7f2ae00b7fd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009d90 tx=0x7f2ad802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a77f5769948cf9ff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:45.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.526+0000 7f2ae5523640 1 -- 172.21.15.52:0/1225099100 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8007740 con 0x7f2ae0162c00 2024-05-11T01:13:45.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.526+0000 7f2ae5523640 1 -- 172.21.15.52:0/1225099100 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad80078e0 con 0x7f2ae0162c00 2024-05-11T01:13:45.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.526+0000 7f2ae5523640 1 -- 172.21.15.52:0/1225099100 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8007c90 con 0x7f2ae0162c00 2024-05-11T01:13:45.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.526+0000 7f2ae8789640 1 -- 172.21.15.52:0/1225099100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae0162c00 msgr2=0x7f2ae00b7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.526+0000 7f2ae8789640 1 --2- 172.21.15.52:0/1225099100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae0162c00 0x7f2ae00b7fd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009d90 tx=0x7f2ad802a6f0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.527+0000 7f2ae8789640 1 -- 172.21.15.52:0/1225099100 shutdown_connections 2024-05-11T01:13:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.527+0000 7f2ae8789640 1 --2- 172.21.15.52:0/1225099100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae0162c00 0x7f2ae00b7fd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.527+0000 7f2ae8789640 1 -- 172.21.15.52:0/1225099100 >> 172.21.15.52:0/1225099100 conn(0x7f2ae009e4f0 msgr2=0x7f2ae00d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.527+0000 7f2ae8789640 1 -- 172.21.15.52:0/1225099100 shutdown_connections 2024-05-11T01:13:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.527+0000 7f2ae8789640 1 -- 172.21.15.52:0/1225099100 wait complete. 2024-05-11T01:13:45.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.528+0000 7f2ae8789640 1 Processor -- start 2024-05-11T01:13:45.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.528+0000 7f2ae8789640 1 -- start start 2024-05-11T01:13:45.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.528+0000 7f2ae8789640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae00d1cc0 0x7f2ae015ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae8789640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ae00d20a0 0x7f2ae009ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae8789640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ae0162c00 0x7f2ae00a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae8789640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00d5cd0 con 0x7f2ae0162c00 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae8789640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00d2fa0 con 0x7f2ae00d1cc0 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae8789640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae00d34c0 con 0x7f2ae00d20a0 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6d26640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ae0162c00 0x7f2ae00a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6525640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae00d1cc0 0x7f2ae015ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6d26640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ae0162c00 0x7f2ae00a26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36342/0 (socket says 172.21.15.52:36342) 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6d26640 1 -- 172.21.15.52:0/435284693 learned_addr learned my addr 172.21.15.52:0/435284693 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae5d24640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ae00d20a0 0x7f2ae009ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6d26640 1 -- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ae00d20a0 msgr2=0x7f2ae009ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6d26640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ae00d20a0 0x7f2ae009ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6d26640 1 -- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae00d1cc0 msgr2=0x7f2ae015ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6d26640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae00d1cc0 0x7f2ae015ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6d26640 1 -- 172.21.15.52:0/435284693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae015c390 con 0x7f2ae0162c00 2024-05-11T01:13:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae6d26640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ae0162c00 0x7f2ae00a26f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ad802ace0 tx=0x7f2ad8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ad6ffd640 1 -- 172.21.15.52:0/435284693 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad803f070 con 0x7f2ae0162c00 2024-05-11T01:13:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ad6ffd640 1 -- 172.21.15.52:0/435284693 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad8031c60 con 0x7f2ae0162c00 2024-05-11T01:13:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ad6ffd640 1 -- 172.21.15.52:0/435284693 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8009450 con 0x7f2ae0162c00 2024-05-11T01:13:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.529+0000 7f2ae8789640 1 -- 172.21.15.52:0/435284693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae015c670 con 0x7f2ae0162c00 2024-05-11T01:13:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.530+0000 7f2ae8789640 1 -- 172.21.15.52:0/435284693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae015cc30 con 0x7f2ae0162c00 2024-05-11T01:13:45.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.531+0000 7f2ae8789640 1 -- 172.21.15.52:0/435284693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae015ce30 con 0x7f2ae0162c00 2024-05-11T01:13:45.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.531+0000 7f2ad6ffd640 1 -- 172.21.15.52:0/435284693 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 98372+0+0 (secure 0 0 0) 0x7f2ad803b410 con 0x7f2ae0162c00 2024-05-11T01:13:45.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.532+0000 7f2ad6ffd640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2ac4076480 0x7f2ac4078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:45.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.532+0000 7f2ae6525640 1 -- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2ac4076480 msgr2=0x7f2ac4078940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/2051242002 2024-05-11T01:13:45.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.532+0000 7f2ad6ffd640 1 -- 172.21.15.52:0/435284693 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6609+0+0 (secure 0 0 0) 0x7f2ad80b8be0 con 0x7f2ae0162c00 2024-05-11T01:13:45.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.532+0000 7f2ae6525640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2ac4076480 0x7f2ac4078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T01:13:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.536+0000 7f2ad6ffd640 1 -- 172.21.15.52:0/435284693 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2ae015ce30 con 0x7f2ae0162c00 2024-05-11T01:13:45.566 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.567+0000 7fca2c5ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T01:13:45.682 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.683+0000 7fca2c5ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T01:13:45.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.692+0000 7f2ad6ffd640 1 -- 172.21.15.52:0/435284693 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 14) ==== 97912+0+0 (secure 0 0 0) 0x7f2ad807da70 con 0x7f2ae0162c00 2024-05-11T01:13:45.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.692+0000 7f2ad6ffd640 1 -- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2ac4076480 msgr2=0x7f2ac4078940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T01:13:45.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.692+0000 7f2ad6ffd640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2ac4076480 0x7f2ac4078940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:45.738 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9b08a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.738 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9b08a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.739 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9b08a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.739 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9b08a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.739 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9ad89a640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.739 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9b08a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.739 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9ad89a640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.739 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9ad89a640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.739 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9ad89a640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.739 INFO:tasks.ceph.mgr.x.smithi119.stderr:2024-05-11T01:13:45.739+0000 7fc9ad89a640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T01:13:45.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:45.771+0000 7f2ae8789640 1 -- 172.21.15.52:0/435284693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2ae00694f0 con 0x7f2ae0162c00 2024-05-11T01:13:45.834 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:45.834+0000 7fbc6cbbf040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T01:13:45.956 INFO:tasks.ceph.mgr.y.smithi052.stderr:2024-05-11T01:13:45.956+0000 7fbc6cbbf040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T01:13:46.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.399+0000 7f2ad6ffd640 1 -- 172.21.15.52:0/435284693 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f2ad809bc40 con 0x7f2ae0162c00 2024-05-11T01:13:46.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 -- 172.21.15.52:0/435284693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ae0162c00 msgr2=0x7f2ae00a26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ae0162c00 0x7f2ae00a26f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ad802ace0 tx=0x7f2ad8004a40 comp rx=0 tx=0).stop 2024-05-11T01:13:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 -- 172.21.15.52:0/435284693 shutdown_connections 2024-05-11T01:13:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:6832/2051242002,v1:172.21.15.119:6833/2051242002] conn(0x7f2ac4076480 0x7f2ac4078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ae0162c00 0x7f2ae00a26f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2ae00d20a0 0x7f2ae009ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 --2- 172.21.15.52:0/435284693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae00d1cc0 0x7f2ae015ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 -- 172.21.15.52:0/435284693 >> 172.21.15.52:0/435284693 conn(0x7f2ae009e4f0 msgr2=0x7f2ae00681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 -- 172.21.15.52:0/435284693 shutdown_connections 2024-05-11T01:13:46.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.401+0000 7f2ae8789640 1 -- 172.21.15.52:0/435284693 wait complete. 2024-05-11T01:13:46.414 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T01:13:46.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.567+0000 7fae380e1640 1 Processor -- start 2024-05-11T01:13:46.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.567+0000 7fae380e1640 1 -- start start 2024-05-11T01:13:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae380e1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fae3015f520 0x7fae300d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae380e1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae300d00f0 0x7fae30070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae380e1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae30072330 con 0x7fae3006d590 2024-05-11T01:13:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae380e1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fae300d1670 con 0x7fae3015f520 2024-05-11T01:13:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae380e1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fae300d1910 con 0x7fae300d00f0 2024-05-11T01:13:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae35e7d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fae3015f520 0x7fae300d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae35e7d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fae3015f520 0x7fae300d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36344/0 (socket says 172.21.15.52:36344) 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae35e7d640 1 -- 172.21.15.52:0/2764299476 learned_addr learned my addr 172.21.15.52:0/2764299476 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae3667e640 1 --2- 172.21.15.52:0/2764299476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae300d00f0 0x7fae30070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae35e7d640 1 -- 172.21.15.52:0/2764299476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae300d00f0 msgr2=0x7fae30070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae35e7d640 1 --2- 172.21.15.52:0/2764299476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae300d00f0 0x7fae30070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae35e7d640 1 -- 172.21.15.52:0/2764299476 >> v1:172.21.15.119:6789/0 conn(0x7fae3006d590 legacy=0x7fae300cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae35e7d640 1 -- 172.21.15.52:0/2764299476 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae300d1bf0 con 0x7fae3015f520 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.568+0000 7fae3667e640 1 --2- 172.21.15.52:0/2764299476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae300d00f0 0x7fae30070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.569+0000 7fae35e7d640 1 --2- 172.21.15.52:0/2764299476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3015f520 0x7fae300d1130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098a0 tx=0x7fae2c02a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef9ce5868cfafc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.569+0000 7fae34e7b640 1 -- 172.21.15.52:0/2764299476 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c037070 con 0x7fae3015f520 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.569+0000 7fae34e7b640 1 -- 172.21.15.52:0/2764299476 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2c032640 con 0x7fae3015f520 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.569+0000 7fae34e7b640 1 -- 172.21.15.52:0/2764299476 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c032a10 con 0x7fae3015f520 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.570+0000 7fae380e1640 1 -- 172.21.15.52:0/2764299476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3015f520 msgr2=0x7fae300d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.570+0000 7fae380e1640 1 --2- 172.21.15.52:0/2764299476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3015f520 0x7fae300d1130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098a0 tx=0x7fae2c02a320 comp rx=0 tx=0).stop 2024-05-11T01:13:46.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.570+0000 7fae380e1640 1 -- 172.21.15.52:0/2764299476 shutdown_connections 2024-05-11T01:13:46.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.570+0000 7fae380e1640 1 --2- 172.21.15.52:0/2764299476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae300d00f0 0x7fae30070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:46.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.570+0000 7fae380e1640 1 --2- 172.21.15.52:0/2764299476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3015f520 0x7fae300d1130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:46.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.570+0000 7fae380e1640 1 -- 172.21.15.52:0/2764299476 >> 172.21.15.52:0/2764299476 conn(0x7fae300d3160 msgr2=0x7fae300d3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:46.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.570+0000 7fae380e1640 1 -- 172.21.15.52:0/2764299476 shutdown_connections 2024-05-11T01:13:46.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.570+0000 7fae380e1640 1 -- 172.21.15.52:0/2764299476 wait complete. 2024-05-11T01:13:46.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.571+0000 7fae380e1640 1 Processor -- start 2024-05-11T01:13:46.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.571+0000 7fae380e1640 1 -- start start 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae380e1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3006d590 0x7fae300a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae380e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae300d00f0 0x7fae3015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae380e1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae3015f520 0x7fae300c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae380e1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae300d2490 con 0x7fae3006d590 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae380e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fae300d21b0 con 0x7fae300d00f0 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae380e1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fae30070c40 con 0x7fae3015f520 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae35e7d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3006d590 0x7fae300a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae35e7d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3006d590 0x7fae300a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36356/0 (socket says 172.21.15.52:36356) 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae35e7d640 1 -- 172.21.15.52:0/3616687117 learned_addr learned my addr 172.21.15.52:0/3616687117 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae3567c640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae300d00f0 0x7fae3015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae3667e640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae3015f520 0x7fae300c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:46.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae35e7d640 1 -- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae3015f520 msgr2=0x7fae300c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae35e7d640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae3015f520 0x7fae300c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae35e7d640 1 -- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae300d00f0 msgr2=0x7fae3015b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae35e7d640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae300d00f0 0x7fae3015b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae35e7d640 1 -- 172.21.15.52:0/3616687117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae300c6fc0 con 0x7fae3006d590 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae35e7d640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3006d590 0x7fae300a09d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098a0 tx=0x7fae2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae267fc640 1 -- 172.21.15.52:0/3616687117 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c041020 con 0x7fae3006d590 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae267fc640 1 -- 172.21.15.52:0/3616687117 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2c030770 con 0x7fae3006d590 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.572+0000 7fae267fc640 1 -- 172.21.15.52:0/3616687117 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c037040 con 0x7fae3006d590 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.573+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae300c72a0 con 0x7fae3006d590 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.573+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae300c7860 con 0x7fae3006d590 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.574+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae30069780 con 0x7fae3006d590 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.574+0000 7fae267fc640 1 -- 172.21.15.52:0/3616687117 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 14) ==== 97912+0+0 (secure 0 0 0) 0x7fae2c044020 con 0x7fae3006d590 2024-05-11T01:13:46.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.575+0000 7fae267fc640 1 -- 172.21.15.52:0/3616687117 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6996+0+0 (secure 0 0 0) 0x7fae2c0b7c40 con 0x7fae3006d590 2024-05-11T01:13:46.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.579+0000 7fae267fc640 1 -- 172.21.15.52:0/3616687117 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fae2c04b030 con 0x7fae3006d590 2024-05-11T01:13:46.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.695+0000 7fae267fc640 1 -- 172.21.15.52:0/3616687117 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 15) ==== 98039+0+0 (secure 0 0 0) 0x7fae2c07c800 con 0x7fae3006d590 2024-05-11T01:13:46.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.695+0000 7fae267fc640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fae1407bfe0 0x7fae1407e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:46.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.697+0000 7fae3567c640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fae1407bfe0 0x7fae1407e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:46.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.702+0000 7fae3567c640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fae1407bfe0 0x7fae1407e3d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae3009e450 tx=0x7fae20009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:46.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:46.776+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fae30160440 con 0x7fae3006d590 2024-05-11T01:13:47.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.699+0000 7fae267fc640 1 -- 172.21.15.52:0/3616687117 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) ==== 181+0+0 (secure 0 0 0) 0x7fae2c080e90 con 0x7fae3006d590 2024-05-11T01:13:47.699 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-05-11T01:13:47.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.702+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fae1407bfe0 msgr2=0x7fae1407e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:47.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fae1407bfe0 0x7fae1407e3d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae3009e450 tx=0x7fae20009040 comp rx=0 tx=0).stop 2024-05-11T01:13:47.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3006d590 msgr2=0x7fae300a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:47.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3006d590 0x7fae300a09d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098a0 tx=0x7fae2c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 shutdown_connections 2024-05-11T01:13:47.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fae1407bfe0 0x7fae1407e3d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae3015f520 0x7fae300c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae300d00f0 0x7fae3015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 --2- 172.21.15.52:0/3616687117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae3006d590 0x7fae300a09d0 secure :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098a0 tx=0x7fae2c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 >> 172.21.15.52:0/3616687117 conn(0x7fae300d3160 msgr2=0x7fae3015f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:47.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 shutdown_connections 2024-05-11T01:13:47.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.703+0000 7fae380e1640 1 -- 172.21.15.52:0/3616687117 wait complete. 2024-05-11T01:13:47.723 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T01:13:47.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.837+0000 7f0959dc9640 1 Processor -- start 2024-05-11T01:13:47.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.838+0000 7f0959dc9640 1 -- start start 2024-05-11T01:13:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.838+0000 7f0959dc9640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f09540d57a0 0x7f09540d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.838+0000 7f0959dc9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09540b4dc0 0x7f09540d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.838+0000 7f0959dc9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09540cfd40 con 0x7f095415f310 2024-05-11T01:13:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.838+0000 7f0959dc9640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f09540b6140 con 0x7f09540d57a0 2024-05-11T01:13:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.838+0000 7f0959dc9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f09540b5530 con 0x7f09540b4dc0 2024-05-11T01:13:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.838+0000 7f0952ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f09540d57a0 0x7f09540d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.839+0000 7f0952ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f09540d57a0 0x7f09540d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36358/0 (socket says 172.21.15.52:36358) 2024-05-11T01:13:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.839+0000 7f0952ffd640 1 -- 172.21.15.52:0/3917768200 learned_addr learned my addr 172.21.15.52:0/3917768200 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.839+0000 7f0953fff640 1 --2- 172.21.15.52:0/3917768200 >> v2:172.21.15.119:3300/0 conn(0x7f09540b4dc0 0x7f09540d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.839+0000 7f0952ffd640 1 -- 172.21.15.52:0/3917768200 >> v2:172.21.15.119:3300/0 conn(0x7f09540b4dc0 msgr2=0x7f09540d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.839+0000 7f0952ffd640 1 --2- 172.21.15.52:0/3917768200 >> v2:172.21.15.119:3300/0 conn(0x7f09540b4dc0 0x7f09540d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.839+0000 7f0952ffd640 1 -- 172.21.15.52:0/3917768200 >> v1:172.21.15.119:6789/0 conn(0x7f095415f310 legacy=0x7f09540d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.839+0000 7f0952ffd640 1 -- 172.21.15.52:0/3917768200 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09540b5810 con 0x7f09540d57a0 2024-05-11T01:13:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f0952ffd640 1 --2- 172.21.15.52:0/3917768200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 0x7f09540d2590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f09540b60b0 tx=0x7f094802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ed932b40ff34e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f09527fc640 1 -- 172.21.15.52:0/3917768200 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948037070 con 0x7f09540d57a0 2024-05-11T01:13:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f09527fc640 1 -- 172.21.15.52:0/3917768200 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0948032440 con 0x7f09540d57a0 2024-05-11T01:13:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f09527fc640 1 -- 172.21.15.52:0/3917768200 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948032810 con 0x7f09540d57a0 2024-05-11T01:13:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f0959dc9640 1 -- 172.21.15.52:0/3917768200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 msgr2=0x7f09540d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f0959dc9640 1 --2- 172.21.15.52:0/3917768200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 0x7f09540d2590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f09540b60b0 tx=0x7f094802a1e0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f0959dc9640 1 -- 172.21.15.52:0/3917768200 shutdown_connections 2024-05-11T01:13:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f0959dc9640 1 --2- 172.21.15.52:0/3917768200 >> v2:172.21.15.119:3300/0 conn(0x7f09540b4dc0 0x7f09540d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f0959dc9640 1 --2- 172.21.15.52:0/3917768200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 0x7f09540d2590 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f0959dc9640 1 -- 172.21.15.52:0/3917768200 >> 172.21.15.52:0/3917768200 conn(0x7f095408eb80 msgr2=0x7f095409e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f0959dc9640 1 -- 172.21.15.52:0/3917768200 shutdown_connections 2024-05-11T01:13:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.840+0000 7f0959dc9640 1 -- 172.21.15.52:0/3917768200 wait complete. 2024-05-11T01:13:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.841+0000 7f0959dc9640 1 Processor -- start 2024-05-11T01:13:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.842+0000 7f0959dc9640 1 -- start start 2024-05-11T01:13:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.842+0000 7f0959dc9640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09540b4dc0 0x7f095414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.842+0000 7f0959dc9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 0x7f09541f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.842+0000 7f0959dc9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095415f310 0x7f09540cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.842+0000 7f0959dc9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f095415f6f0 con 0x7f09540d57a0 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.842+0000 7f0959dc9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09540b5dd0 con 0x7f095415f310 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.842+0000 7f0959dc9640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f09540b51d0 con 0x7f09540b4dc0 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0952ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 0x7f09541f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0952ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 0x7f09541f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36362/0 (socket says 172.21.15.52:36362) 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0952ffd640 1 -- 172.21.15.52:0/3677298486 learned_addr learned my addr 172.21.15.52:0/3677298486 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0953fff640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095415f310 0x7f09540cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f09537fe640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09540b4dc0 0x7f095414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0952ffd640 1 -- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09540b4dc0 msgr2=0x7f095414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0952ffd640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09540b4dc0 0x7f095414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0952ffd640 1 -- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095415f310 msgr2=0x7f09540cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0952ffd640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095415f310 0x7f09540cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0952ffd640 1 -- 172.21.15.52:0/3677298486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09541605a0 con 0x7f09540d57a0 2024-05-11T01:13:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0953fff640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095415f310 0x7f09540cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f09537fe640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09540b4dc0 0x7f095414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0952ffd640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 0x7f09541f6060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0948000c00 tx=0x7f09480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f093bfff640 1 -- 172.21.15.52:0/3677298486 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948041020 con 0x7f09540d57a0 2024-05-11T01:13:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f093bfff640 1 -- 172.21.15.52:0/3677298486 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0948030970 con 0x7f09540d57a0 2024-05-11T01:13:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09540c9e50 con 0x7f09540d57a0 2024-05-11T01:13:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f093bfff640 1 -- 172.21.15.52:0/3677298486 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948037040 con 0x7f09540d57a0 2024-05-11T01:13:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.843+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09540ca410 con 0x7f09540d57a0 2024-05-11T01:13:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.844+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0954069780 con 0x7f09540d57a0 2024-05-11T01:13:47.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.845+0000 7f093bfff640 1 -- 172.21.15.52:0/3677298486 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 15) ==== 98039+0+0 (secure 0 0 0) 0x7f0948044020 con 0x7f09540d57a0 2024-05-11T01:13:47.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.845+0000 7f093bfff640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0940076120 0x7f09400785e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:47.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.846+0000 7f093bfff640 1 -- 172.21.15.52:0/3677298486 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6558+0+0 (secure 0 0 0) 0x7f09480b8000 con 0x7f09540d57a0 2024-05-11T01:13:47.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.846+0000 7f09537fe640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0940076120 0x7f09400785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:47.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.846+0000 7f09537fe640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0940076120 0x7f09400785e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0944006fd0 tx=0x7f0944008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:47.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:47.850+0000 7f093bfff640 1 -- 172.21.15.52:0/3677298486 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f094804b030 con 0x7f09540d57a0 2024-05-11T01:13:48.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:48.030+0000 7f093bfff640 1 -- 172.21.15.52:0/3677298486 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 16) ==== 98145+0+0 (secure 0 0 0) 0x7f094807d460 con 0x7f09540d57a0 2024-05-11T01:13:48.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:48.044+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f095414ba00 con 0x7f09540d57a0 2024-05-11T01:13:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.028+0000 7f093bfff640 1 -- 172.21.15.52:0/3677298486 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) ==== 169+0+0 (secure 0 0 0) 0x7f09540cfd40 con 0x7f09540d57a0 2024-05-11T01:13:49.029 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-05-11T01:13:49.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0940076120 msgr2=0x7f09400785e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0940076120 0x7f09400785e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0944006fd0 tx=0x7f0944008040 comp rx=0 tx=0).stop 2024-05-11T01:13:49.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 msgr2=0x7f09541f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 0x7f09541f6060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0948000c00 tx=0x7f09480046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 shutdown_connections 2024-05-11T01:13:49.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0940076120 0x7f09400785e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f095415f310 0x7f09540cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09540d57a0 0x7f09541f6060 secure :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0948000c00 tx=0x7f09480046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 --2- 172.21.15.52:0/3677298486 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09540b4dc0 0x7f095414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 >> 172.21.15.52:0/3677298486 conn(0x7f095408eb80 msgr2=0x7f095406e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:49.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 shutdown_connections 2024-05-11T01:13:49.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.032+0000 7f0959dc9640 1 -- 172.21.15.52:0/3677298486 wait complete. 2024-05-11T01:13:49.048 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-05-11T01:13:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.160+0000 7f7a856e8640 1 Processor -- start 2024-05-11T01:13:49.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.160+0000 7f7a856e8640 1 -- start start 2024-05-11T01:13:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a856e8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7a8009c130 0x7f7a8006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a856e8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8006e140 con 0x7f7a800d2b40 2024-05-11T01:13:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a856e8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8006e420 con 0x7f7a800b4320 2024-05-11T01:13:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a856e8640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8006dd80 con 0x7f7a8009c130 2024-05-11T01:13:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a7f7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7a8009c130 0x7f7a8006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a7f7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7a8009c130 0x7f7a8006d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36374/0 (socket says 172.21.15.52:36374) 2024-05-11T01:13:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a7f7fe640 1 -- 172.21.15.52:0/926231500 learned_addr learned my addr 172.21.15.52:0/926231500 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a7f7fe640 1 -- 172.21.15.52:0/926231500 >> v1:172.21.15.119:6789/0 conn(0x7f7a800b4320 legacy=0x7f7a8009ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a7f7fe640 1 -- 172.21.15.52:0/926231500 >> v1:172.21.15.52:6789/0 conn(0x7f7a800d2b40 legacy=0x7f7a800b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.161+0000 7f7a7f7fe640 1 -- 172.21.15.52:0/926231500 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a800d1e60 con 0x7f7a8009c130 2024-05-11T01:13:49.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.162+0000 7f7a7f7fe640 1 --2- 172.21.15.52:0/926231500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a8009c130 0x7f7a8006d7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e50 tx=0x7f7a7802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6255df3c5f9571 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:49.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.162+0000 7f7a7dffb640 1 -- 172.21.15.52:0/926231500 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78036070 con 0x7f7a8009c130 2024-05-11T01:13:49.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.162+0000 7f7a7dffb640 1 -- 172.21.15.52:0/926231500 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a78031600 con 0x7f7a8009c130 2024-05-11T01:13:49.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.162+0000 7f7a7dffb640 1 -- 172.21.15.52:0/926231500 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a780319d0 con 0x7f7a8009c130 2024-05-11T01:13:49.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.163+0000 7f7a856e8640 1 -- 172.21.15.52:0/926231500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a8009c130 msgr2=0x7f7a8006d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.163+0000 7f7a856e8640 1 --2- 172.21.15.52:0/926231500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a8009c130 0x7f7a8006d7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e50 tx=0x7f7a7802a680 comp rx=0 tx=0).stop 2024-05-11T01:13:49.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.163+0000 7f7a856e8640 1 -- 172.21.15.52:0/926231500 shutdown_connections 2024-05-11T01:13:49.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.163+0000 7f7a856e8640 1 --2- 172.21.15.52:0/926231500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a8009c130 0x7f7a8006d7a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.163+0000 7f7a856e8640 1 -- 172.21.15.52:0/926231500 >> 172.21.15.52:0/926231500 conn(0x7f7a800b4a60 msgr2=0x7f7a800b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:49.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.163+0000 7f7a856e8640 1 -- 172.21.15.52:0/926231500 shutdown_connections 2024-05-11T01:13:49.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.163+0000 7f7a856e8640 1 -- 172.21.15.52:0/926231500 wait complete. 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.164+0000 7f7a856e8640 1 Processor -- start 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.164+0000 7f7a856e8640 1 -- start start 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a856e8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a8009c130 0x7f7a8014c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a856e8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800b4320 0x7f7a800b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a856e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a800d2b40 0x7f7a801620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a856e8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800b5980 con 0x7f7a800b4320 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a856e8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800b5800 con 0x7f7a800d2b40 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a856e8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7a800d2380 con 0x7f7a8009c130 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800b4320 0x7f7a800b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800b4320 0x7f7a800b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36382/0 (socket says 172.21.15.52:36382) 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7e7fc640 1 -- 172.21.15.52:0/3592805381 learned_addr learned my addr 172.21.15.52:0/3592805381 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7effd640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a8009c130 0x7f7a8014c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7f7fe640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a800d2b40 0x7f7a801620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7e7fc640 1 -- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a8009c130 msgr2=0x7f7a8014c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7e7fc640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a8009c130 0x7f7a8014c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7e7fc640 1 -- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a800d2b40 msgr2=0x7f7a801620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7e7fc640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a800d2b40 0x7f7a801620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7e7fc640 1 -- 172.21.15.52:0/3592805381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a800b7680 con 0x7f7a800b4320 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a7e7fc640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800b4320 0x7f7a800b6be0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c3a0 tx=0x7f7a7400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a677fe640 1 -- 172.21.15.52:0/3592805381 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74017070 con 0x7f7a800b4320 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a677fe640 1 -- 172.21.15.52:0/3592805381 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a740052d0 con 0x7f7a800b4320 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.165+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a800b7960 con 0x7f7a800b4320 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.166+0000 7f7a677fe640 1 -- 172.21.15.52:0/3592805381 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74003830 con 0x7f7a800b4320 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.166+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a800b8200 con 0x7f7a800b4320 2024-05-11T01:13:49.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.167+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a800b7b30 con 0x7f7a800b4320 2024-05-11T01:13:49.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.167+0000 7f7a677fe640 1 -- 172.21.15.52:0/3592805381 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 98145+0+0 (secure 0 0 0) 0x7f7a7401a070 con 0x7f7a800b4320 2024-05-11T01:13:49.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.168+0000 7f7a677fe640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7a6c076240 0x7f7a6c078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.168+0000 7f7a677fe640 1 -- 172.21.15.52:0/3592805381 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7f7a74096a00 con 0x7f7a800b4320 2024-05-11T01:13:49.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.168+0000 7f7a7effd640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7a6c076240 0x7f7a6c078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.169+0000 7f7a7effd640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7a6c076240 0x7f7a6c078700 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a70003d90 tx=0x7f7a70009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:49.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.172+0000 7f7a677fe640 1 -- 172.21.15.52:0/3592805381 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7a74060290 con 0x7f7a800b4320 2024-05-11T01:13:49.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.370+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) -- 0x7f7a800695d0 con 0x7f7a800b4320 2024-05-11T01:13:49.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.373+0000 7f7a677fe640 1 -- 172.21.15.52:0/3592805381 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v13) ==== 120+0+0 (secure 0 0 0) 0x7f7a7405fb90 con 0x7f7a800b4320 2024-05-11T01:13:49.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7a6c076240 msgr2=0x7f7a6c078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7a6c076240 0x7f7a6c078700 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a70003d90 tx=0x7f7a70009040 comp rx=0 tx=0).stop 2024-05-11T01:13:49.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800b4320 msgr2=0x7f7a800b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800b4320 0x7f7a800b6be0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c3a0 tx=0x7f7a7400c860 comp rx=0 tx=0).stop 2024-05-11T01:13:49.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 shutdown_connections 2024-05-11T01:13:49.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7a6c076240 0x7f7a6c078700 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a800d2b40 0x7f7a801620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a800b4320 0x7f7a800b6be0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 --2- 172.21.15.52:0/3592805381 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a8009c130 0x7f7a8014c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 >> 172.21.15.52:0/3592805381 conn(0x7f7a800b4a60 msgr2=0x7f7a8009ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:49.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 shutdown_connections 2024-05-11T01:13:49.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.376+0000 7f7a856e8640 1 -- 172.21.15.52:0/3592805381 wait complete. 2024-05-11T01:13:49.387 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2024-05-11T01:13:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.492+0000 7fbcde1af640 1 Processor -- start 2024-05-11T01:13:49.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.492+0000 7fbcde1af640 1 -- start start 2024-05-11T01:13:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcde1af640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbcd815f430 0x7fbcd80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcde1af640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcd80b5420 0x7fbcd80d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcde1af640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd80cfc60 con 0x7fbcd80d1eb0 2024-05-11T01:13:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcde1af640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd80d2a80 con 0x7fbcd815f430 2024-05-11T01:13:49.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcde1af640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd8163120 con 0x7fbcd80b5420 2024-05-11T01:13:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcd7fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fbcd80d1eb0 0x7fbcd80d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36386/0 (socket says 172.21.15.52:36386) 2024-05-11T01:13:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcd7fff640 1 -- 172.21.15.52:0/1984668399 learned_addr learned my addr 172.21.15.52:0/1984668399 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcd6ffd640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcd80b5420 0x7fbcd80d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcd77fe640 1 --2- 172.21.15.52:0/1984668399 >> v2:172.21.15.119:3300/0 conn(0x7fbcd815f430 0x7fbcd80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 283036896 0 0) 0x7fbcd80cfc60 con 0x7fbcd80d1eb0 2024-05-11T01:13:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.493+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcc0003520 con 0x7fbcd80d1eb0 2024-05-11T01:13:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 607523269 0 0) 0x7fbcc0003520 con 0x7fbcd80d1eb0 2024-05-11T01:13:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcd80b5420 msgr2=0x7fbcd80d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:49.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcd80b5420 0x7fbcd80d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 >> v2:172.21.15.119:3300/0 conn(0x7fbcd815f430 msgr2=0x7fbcd80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:49.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 --2- 172.21.15.52:0/1984668399 >> v2:172.21.15.119:3300/0 conn(0x7fbcd815f430 0x7fbcd80b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd80d24c0 con 0x7fbcd80d1eb0 2024-05-11T01:13:49.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fbcd0001a70 con 0x7fbcd80d1eb0 2024-05-11T01:13:49.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd6ffd640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcd80b5420 0x7fbcd80d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:13:49.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 >> v1:172.21.15.52:6789/0 conn(0x7fbcd80d1eb0 legacy=0x7fbcd80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a00 0x7fbcc0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcd80b5420 0x7fbcc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcc0004d40 0x7fbcc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd80d2a80 con 0x7fbcc0004d40 2024-05-11T01:13:49.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8163120 con 0x7fbcc0003a00 2024-05-11T01:13:49.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc000b9c0 con 0x7fbcd80b5420 2024-05-11T01:13:49.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd7fff640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcc0004d40 0x7fbcc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd77fe640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a00 0x7fbcc0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd6ffd640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcd80b5420 0x7fbcc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd7fff640 1 -- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcd80b5420 msgr2=0x7fbcc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:49.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd7fff640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcd80b5420 0x7fbcc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd7fff640 1 -- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a00 msgr2=0x7fbcc0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:49.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd7fff640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a00 0x7fbcc0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.494+0000 7fbcd7fff640 1 -- 172.21.15.52:0/1984668399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc0003520 con 0x7fbcc0004d40 2024-05-11T01:13:49.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.495+0000 7fbcd7fff640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcc0004d40 0x7fbcc00051c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000a8a0 tx=0x7fbcc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe6526c74c3b81 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:49.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.495+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000a4c0 con 0x7fbcc0004d40 2024-05-11T01:13:49.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.495+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd002f070 con 0x7fbcc0004d40 2024-05-11T01:13:49.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.495+0000 7fbcd67fc640 1 -- 172.21.15.52:0/1984668399 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd002bc60 con 0x7fbcc0004d40 2024-05-11T01:13:49.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.495+0000 7fbcde1af640 1 -- 172.21.15.52:0/1984668399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcc0004d40 msgr2=0x7fbcc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.495+0000 7fbcde1af640 1 --2- 172.21.15.52:0/1984668399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcc0004d40 0x7fbcc00051c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000a8a0 tx=0x7fbcc0005830 comp rx=0 tx=0).stop 2024-05-11T01:13:49.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.495+0000 7fbcd7fff640 1 -- 172.21.15.52:0/1984668399 reap_dead start 2024-05-11T01:13:49.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.496+0000 7fbcde1af640 1 -- 172.21.15.52:0/1984668399 shutdown_connections 2024-05-11T01:13:49.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.496+0000 7fbcde1af640 1 -- 172.21.15.52:0/1984668399 >> 172.21.15.52:0/1984668399 conn(0x7fbcd808ebb0 msgr2=0x7fbcd809e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:49.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.496+0000 7fbcde1af640 1 -- 172.21.15.52:0/1984668399 shutdown_connections 2024-05-11T01:13:49.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.496+0000 7fbcde1af640 1 -- 172.21.15.52:0/1984668399 wait complete. 2024-05-11T01:13:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.497+0000 7fbcde1af640 1 Processor -- start 2024-05-11T01:13:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.497+0000 7fbcde1af640 1 -- start start 2024-05-11T01:13:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcde1af640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcc0004d40 0x7fbcd81f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcde1af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd80b7d80 0x7fbcd806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcde1af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd806e1d0 0x7fbcd8161750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcde1af640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd80d01e0 con 0x7fbcd80b7d80 2024-05-11T01:13:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcde1af640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd80cff00 con 0x7fbcd806e1d0 2024-05-11T01:13:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcde1af640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd80d0e10 con 0x7fbcc0004d40 2024-05-11T01:13:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd80b7d80 0x7fbcd806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd80b7d80 0x7fbcd806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36392/0 (socket says 172.21.15.52:36392) 2024-05-11T01:13:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd6ffd640 1 -- 172.21.15.52:0/1166220702 learned_addr learned my addr 172.21.15.52:0/1166220702 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd77fe640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcc0004d40 0x7fbcd81f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd7fff640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd806e1d0 0x7fbcd8161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd6ffd640 1 -- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcc0004d40 msgr2=0x7fbcd81f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd6ffd640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcc0004d40 0x7fbcd81f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd6ffd640 1 -- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd806e1d0 msgr2=0x7fbcd8161750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd6ffd640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd806e1d0 0x7fbcd8161750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd6ffd640 1 -- 172.21.15.52:0/1166220702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd81f4fd0 con 0x7fbcd80b7d80 2024-05-11T01:13:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd7fff640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd806e1d0 0x7fbcd8161750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcd6ffd640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd80b7d80 0x7fbcd806dc90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbccc011990 tx=0x7fbccc011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.498+0000 7fbcbffff640 1 -- 172.21.15.52:0/1166220702 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00fad0 con 0x7fbcd80b7d80 2024-05-11T01:13:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.499+0000 7fbcbffff640 1 -- 172.21.15.52:0/1166220702 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbccc00fc70 con 0x7fbcd80b7d80 2024-05-11T01:13:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.499+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd81624f0 con 0x7fbcd80b7d80 2024-05-11T01:13:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.499+0000 7fbcbffff640 1 -- 172.21.15.52:0/1166220702 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc004030 con 0x7fbcd80b7d80 2024-05-11T01:13:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.499+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd80c92f0 con 0x7fbcd80b7d80 2024-05-11T01:13:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.500+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd80695d0 con 0x7fbcd80b7d80 2024-05-11T01:13:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.500+0000 7fbcbffff640 1 -- 172.21.15.52:0/1166220702 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 98145+0+0 (secure 0 0 0) 0x7fbccc012040 con 0x7fbcd80b7d80 2024-05-11T01:13:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.501+0000 7fbcbffff640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fbcc4076270 0x7fbcc4078730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.501+0000 7fbcbffff640 1 -- 172.21.15.52:0/1166220702 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7fbccc097970 con 0x7fbcd80b7d80 2024-05-11T01:13:49.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.501+0000 7fbcd77fe640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fbcc4076270 0x7fbcc4078730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.502+0000 7fbcd77fe640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fbcc4076270 0x7fbcc4078730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009b00 tx=0x7fbcc8008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:49.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.505+0000 7fbcbffff640 1 -- 172.21.15.52:0/1166220702 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbccc019070 con 0x7fbcd80b7d80 2024-05-11T01:13:49.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.694+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) -- 0x7fbcd806e5b0 con 0x7fbcd80b7d80 2024-05-11T01:13:49.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.697+0000 7fbcbffff640 1 -- 172.21.15.52:0/1166220702 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) ==== 115+0+0 (secure 0 0 0) 0x7fbccc061200 con 0x7fbcd80b7d80 2024-05-11T01:13:49.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fbcc4076270 msgr2=0x7fbcc4078730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fbcc4076270 0x7fbcc4078730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009b00 tx=0x7fbcc8008040 comp rx=0 tx=0).stop 2024-05-11T01:13:49.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd80b7d80 msgr2=0x7fbcd806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd80b7d80 0x7fbcd806dc90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbccc011990 tx=0x7fbccc011e50 comp rx=0 tx=0).stop 2024-05-11T01:13:49.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 shutdown_connections 2024-05-11T01:13:49.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fbcc4076270 0x7fbcc4078730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcd806e1d0 0x7fbcd8161750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd80b7d80 0x7fbcd806dc90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 --2- 172.21.15.52:0/1166220702 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbcc0004d40 0x7fbcd81f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 >> 172.21.15.52:0/1166220702 conn(0x7fbcd808ebb0 msgr2=0x7fbcd80b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:49.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 shutdown_connections 2024-05-11T01:13:49.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.700+0000 7fbcde1af640 1 -- 172.21.15.52:0/1166220702 wait complete. 2024-05-11T01:13:49.711 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-11T01:13:49.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.816+0000 7f69588bd640 1 Processor -- start 2024-05-11T01:13:49.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.816+0000 7f69588bd640 1 -- start start 2024-05-11T01:13:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f69588bd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69540b3cf0 0x7f69540d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f69588bd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69540d33e0 0x7f695406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f69588bd640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6954160890 con 0x7f695415cdd0 2024-05-11T01:13:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f69588bd640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f69540d3850 con 0x7f69540b3cf0 2024-05-11T01:13:49.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f69588bd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f69540b2c50 con 0x7f69540d33e0 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f695259c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f695415cdd0 0x7f6954160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36402/0 (socket says 172.21.15.52:36402) 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f695259c640 1 -- 172.21.15.52:0/460429448 learned_addr learned my addr 172.21.15.52:0/460429448 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f6952d9d640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69540d33e0 0x7f695406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2289858951 0 0) 0x7f6954160890 con 0x7f695415cdd0 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f6951d9b640 1 --2- 172.21.15.52:0/460429448 >> v2:172.21.15.119:3300/0 conn(0x7f69540b3cf0 0x7f69540d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.817+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6940003520 con 0x7f695415cdd0 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4214702692 0 0) 0x7f6940003520 con 0x7f695415cdd0 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69540d33e0 msgr2=0x7f695406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69540d33e0 0x7f695406d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 >> v2:172.21.15.119:3300/0 conn(0x7f69540b3cf0 msgr2=0x7f69540d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 --2- 172.21.15.52:0/460429448 >> v2:172.21.15.119:3300/0 conn(0x7f69540b3cf0 0x7f69540d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69540b2f30 con 0x7f695415cdd0 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f694c001a70 con 0x7f695415cdd0 2024-05-11T01:13:49.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 >> v1:172.21.15.52:6789/0 conn(0x7f695415cdd0 legacy=0x7f6954160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69540d33e0 0x7f6940003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940003fd0 0x7f6940004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6940004d40 0x7f69400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69540d3850 con 0x7f6940004d40 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69540b2c50 con 0x7f6940003fd0 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f694000ba10 con 0x7f69540d33e0 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f6952d9d640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6940004d40 0x7f69400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f6951d9b640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940003fd0 0x7f6940004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f695259c640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69540d33e0 0x7f6940003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f6952d9d640 1 -- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69540d33e0 msgr2=0x7f6940003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f6952d9d640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69540d33e0 0x7f6940003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f6952d9d640 1 -- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940003fd0 msgr2=0x7f6940004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:49.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f6952d9d640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940003fd0 0x7f6940004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.818+0000 7f6952d9d640 1 -- 172.21.15.52:0/460429448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6940003520 con 0x7f6940004d40 2024-05-11T01:13:49.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.819+0000 7f6952d9d640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6940004d40 0x7f69400051c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6944009af0 tx=0x7f6940005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368d2c33826ddca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:49.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.819+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944036070 con 0x7f6940004d40 2024-05-11T01:13:49.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.819+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6944031400 con 0x7f6940004d40 2024-05-11T01:13:49.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.819+0000 7f695159a640 1 -- 172.21.15.52:0/460429448 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944009690 con 0x7f6940004d40 2024-05-11T01:13:49.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.820+0000 7f69588bd640 1 -- 172.21.15.52:0/460429448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6940004d40 msgr2=0x7f69400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.820+0000 7f69588bd640 1 --2- 172.21.15.52:0/460429448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6940004d40 0x7f69400051c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6944009af0 tx=0x7f6940005830 comp rx=0 tx=0).stop 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.820+0000 7f6952d9d640 1 -- 172.21.15.52:0/460429448 reap_dead start 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.820+0000 7f69588bd640 1 -- 172.21.15.52:0/460429448 shutdown_connections 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.820+0000 7f69588bd640 1 -- 172.21.15.52:0/460429448 >> 172.21.15.52:0/460429448 conn(0x7f695409ba20 msgr2=0x7f695409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.820+0000 7f69588bd640 1 -- 172.21.15.52:0/460429448 shutdown_connections 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.820+0000 7f69588bd640 1 -- 172.21.15.52:0/460429448 wait complete. 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.821+0000 7f69588bd640 1 Processor -- start 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.821+0000 7f69588bd640 1 -- start start 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f69588bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940004d40 0x7f695414c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f69588bd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69541e57b0 0x7f695415dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f69588bd640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69541e5c30 0x7f695415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f69588bd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69540b37d0 con 0x7f69541e57b0 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f69588bd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69540b34f0 con 0x7f6940004d40 2024-05-11T01:13:49.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f69588bd640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f695415d1b0 con 0x7f69541e5c30 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f6951d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69541e57b0 0x7f695415dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f6951d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69541e57b0 0x7f695415dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36408/0 (socket says 172.21.15.52:36408) 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f6951d9b640 1 -- 172.21.15.52:0/1248229400 learned_addr learned my addr 172.21.15.52:0/1248229400 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f6952d9d640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69541e5c30 0x7f695415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f695259c640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940004d40 0x7f695414c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f6951d9b640 1 -- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69541e5c30 msgr2=0x7f695415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f6951d9b640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69541e5c30 0x7f695415e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f6951d9b640 1 -- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940004d40 msgr2=0x7f695414c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f6951d9b640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940004d40 0x7f695414c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f6951d9b640 1 -- 172.21.15.52:0/1248229400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69541e6100 con 0x7f69541e57b0 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.822+0000 7f695259c640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940004d40 0x7f695414c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.823+0000 7f6951d9b640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69541e57b0 0x7f695415dae0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f693c011960 tx=0x7f693c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:49.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.823+0000 7f694affd640 1 -- 172.21.15.52:0/1248229400 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c00faf0 con 0x7f69541e57b0 2024-05-11T01:13:49.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.823+0000 7f694affd640 1 -- 172.21.15.52:0/1248229400 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f693c00fc90 con 0x7f69541e57b0 2024-05-11T01:13:49.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.823+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69541e63e0 con 0x7f69541e57b0 2024-05-11T01:13:49.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.823+0000 7f694affd640 1 -- 172.21.15.52:0/1248229400 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c014440 con 0x7f69541e57b0 2024-05-11T01:13:49.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.823+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541e69a0 con 0x7f69541e57b0 2024-05-11T01:13:49.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.824+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69540694f0 con 0x7f69541e57b0 2024-05-11T01:13:49.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.824+0000 7f694affd640 1 -- 172.21.15.52:0/1248229400 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 98145+0+0 (secure 0 0 0) 0x7f693c012040 con 0x7f69541e57b0 2024-05-11T01:13:49.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.825+0000 7f694affd640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6934076240 0x7f6934078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:49.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.825+0000 7f694affd640 1 -- 172.21.15.52:0/1248229400 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7f693c097d30 con 0x7f69541e57b0 2024-05-11T01:13:49.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.825+0000 7f695259c640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6934076240 0x7f6934078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:49.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.826+0000 7f695259c640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6934076240 0x7f6934078700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f694c004da0 tx=0x7f694c006040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:49.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:49.829+0000 7f694affd640 1 -- 172.21.15.52:0/1248229400 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f693c09c070 con 0x7f69541e57b0 2024-05-11T01:13:50.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.020+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f69540b2990 con 0x7f69541e57b0 2024-05-11T01:13:50.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.023+0000 7f694affd640 1 -- 172.21.15.52:0/1248229400 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f693c019030 con 0x7f69541e57b0 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6934076240 msgr2=0x7f6934078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6934076240 0x7f6934078700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f694c004da0 tx=0x7f694c006040 comp rx=0 tx=0).stop 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69541e57b0 msgr2=0x7f695415dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69541e57b0 0x7f695415dae0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f693c011960 tx=0x7f693c011e20 comp rx=0 tx=0).stop 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 shutdown_connections 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6934076240 0x7f6934078700 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69541e5c30 0x7f695415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69541e57b0 0x7f695415dae0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 --2- 172.21.15.52:0/1248229400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940004d40 0x7f695414c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 >> 172.21.15.52:0/1248229400 conn(0x7f695409ba20 msgr2=0x7f6954071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 shutdown_connections 2024-05-11T01:13:50.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.025+0000 7f69588bd640 1 -- 172.21.15.52:0/1248229400 wait complete. 2024-05-11T01:13:50.037 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2024-05-11T01:13:50.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.140+0000 7ffba3247640 1 Processor -- start 2024-05-11T01:13:50.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.140+0000 7ffba3247640 1 -- start start 2024-05-11T01:13:50.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.141+0000 7ffba3247640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:50.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.141+0000 7ffba3247640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb9c071470 0x7ffb9c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:50.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.141+0000 7ffba3247640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9c0d1240 con 0x7ffb9c15f420 2024-05-11T01:13:50.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.141+0000 7ffba3247640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ffb9c15f800 con 0x7ffb9c071470 2024-05-11T01:13:50.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.141+0000 7ffba3247640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9c06df90 con 0x7ffb9c0b2400 2024-05-11T01:13:50.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.141+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:50.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.141+0000 7ffba17e4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb9c071470 0x7ffb9c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:50.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.142+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.52:33378/0 (socket says 172.21.15.52:33378) 2024-05-11T01:13:50.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.142+0000 7ffb9bfff640 1 -- 172.21.15.52:0/3599338442 learned_addr learned my addr 172.21.15.52:0/3599338442 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:50.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.142+0000 7ffba0fe3640 1 --1- 172.21.15.52:0/3599338442 >> v1:172.21.15.119:6789/0 conn(0x7ffb9c15f420 0x7ffb9c06d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:54536/0 (socket says 172.21.15.52:54536) 2024-05-11T01:13:50.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.142+0000 7ffb9bfff640 1 -- 172.21.15.52:0/3599338442 >> v2:172.21.15.119:3300/0 conn(0x7ffb9c071470 msgr2=0x7ffb9c0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:50.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.142+0000 7ffb9bfff640 1 --2- 172.21.15.52:0/3599338442 >> v2:172.21.15.119:3300/0 conn(0x7ffb9c071470 0x7ffb9c0cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:50.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.142+0000 7ffb9bfff640 1 -- 172.21.15.52:0/3599338442 >> v1:172.21.15.119:6789/0 conn(0x7ffb9c15f420 legacy=0x7ffb9c06d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:50.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.142+0000 7ffb9bfff640 1 -- 172.21.15.52:0/3599338442 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c0d14b0 con 0x7ffb9c0b2400 2024-05-11T01:13:50.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffb9bfff640 1 --2- 172.21.15.52:0/3599338442 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c070f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009870 tx=0x7ffb8402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe37a337f4d131e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:50.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffb9b7fe640 1 -- 172.21.15.52:0/3599338442 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84037070 con 0x7ffb9c0b2400 2024-05-11T01:13:50.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffb9b7fe640 1 -- 172.21.15.52:0/3599338442 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb84032440 con 0x7ffb9c0b2400 2024-05-11T01:13:50.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffb9b7fe640 1 -- 172.21.15.52:0/3599338442 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84032810 con 0x7ffb9c0b2400 2024-05-11T01:13:50.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffba3247640 1 -- 172.21.15.52:0/3599338442 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 msgr2=0x7ffb9c070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:50.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffba3247640 1 --2- 172.21.15.52:0/3599338442 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c070f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009870 tx=0x7ffb8402a270 comp rx=0 tx=0).stop 2024-05-11T01:13:50.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffba3247640 1 -- 172.21.15.52:0/3599338442 shutdown_connections 2024-05-11T01:13:50.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffba3247640 1 --2- 172.21.15.52:0/3599338442 >> v2:172.21.15.119:3300/0 conn(0x7ffb9c071470 0x7ffb9c0cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:50.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffba3247640 1 --2- 172.21.15.52:0/3599338442 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c070f30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:50.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffba3247640 1 -- 172.21.15.52:0/3599338442 >> 172.21.15.52:0/3599338442 conn(0x7ffb9c0d3080 msgr2=0x7ffb9c0d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:50.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffba3247640 1 -- 172.21.15.52:0/3599338442 shutdown_connections 2024-05-11T01:13:50.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.143+0000 7ffba3247640 1 -- 172.21.15.52:0/3599338442 wait complete. 2024-05-11T01:13:50.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.145+0000 7ffba3247640 1 Processor -- start 2024-05-11T01:13:50.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.145+0000 7ffba3247640 1 -- start start 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.145+0000 7ffba3247640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c071470 0x7ffb9c155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba3247640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba3247640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb9c15f420 0x7ffb9c0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba3247640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c0d1d50 con 0x7ffb9c071470 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba3247640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c0d1a70 con 0x7ffb9c15f420 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba3247640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9c0cff60 con 0x7ffb9c0b2400 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba0fe3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c071470 0x7ffb9c155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba0fe3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c071470 0x7ffb9c155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36420/0 (socket says 172.21.15.52:36420) 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba0fe3640 1 -- 172.21.15.52:0/1337338977 learned_addr learned my addr 172.21.15.52:0/1337338977 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffb9bfff640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba17e4640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb9c15f420 0x7ffb9c0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba0fe3640 1 -- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 msgr2=0x7ffb9c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba0fe3640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c0ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:50.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba0fe3640 1 -- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb9c15f420 msgr2=0x7ffb9c0ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:50.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba0fe3640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb9c15f420 0x7ffb9c0ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:50.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba0fe3640 1 -- 172.21.15.52:0/1337338977 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c0aff80 con 0x7ffb9c071470 2024-05-11T01:13:50.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba0fe3640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c071470 0x7ffb9c155420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400c400 tx=0x7ffb9400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:50.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffb997fa640 1 -- 172.21.15.52:0/1337338977 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb940074e0 con 0x7ffb9c071470 2024-05-11T01:13:50.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffb997fa640 1 -- 172.21.15.52:0/1337338977 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb94007680 con 0x7ffb9c071470 2024-05-11T01:13:50.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9c0b0260 con 0x7ffb9c071470 2024-05-11T01:13:50.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffb997fa640 1 -- 172.21.15.52:0/1337338977 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb940049e0 con 0x7ffb9c071470 2024-05-11T01:13:50.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.146+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9c155960 con 0x7ffb9c071470 2024-05-11T01:13:50.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.147+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb9c0696a0 con 0x7ffb9c071470 2024-05-11T01:13:50.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.148+0000 7ffb997fa640 1 -- 172.21.15.52:0/1337338977 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 98145+0+0 (secure 0 0 0) 0x7ffb9401a020 con 0x7ffb9c071470 2024-05-11T01:13:50.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.149+0000 7ffb997fa640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ffb7c076240 0x7ffb7c078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:50.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.149+0000 7ffb997fa640 1 -- 172.21.15.52:0/1337338977 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7ffb94096cf0 con 0x7ffb9c071470 2024-05-11T01:13:50.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.149+0000 7ffb9bfff640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ffb7c076240 0x7ffb7c078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:50.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.149+0000 7ffb9bfff640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ffb7c076240 0x7ffb7c078700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009870 tx=0x7ffb8402e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:50.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.153+0000 7ffb997fa640 1 -- 172.21.15.52:0/1337338977 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffb9405ff30 con 0x7ffb9c071470 2024-05-11T01:13:50.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.377+0000 7ffb997fa640 1 -- 172.21.15.52:0/1337338977 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7ffb9405c1c0 con 0x7ffb9c071470 2024-05-11T01:13:50.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:50.425+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) -- 0x7ffb9c09ba20 con 0x7ffb9c071470 2024-05-11T01:13:51.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.036+0000 7ffb997fa640 1 -- 172.21.15.52:0/1337338977 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v443) ==== 158+0+0 (secure 0 0 0) 0x7ffb94062030 con 0x7ffb9c071470 2024-05-11T01:13:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.039+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ffb7c076240 msgr2=0x7ffb7c078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.039+0000 7ffba3247640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ffb7c076240 0x7ffb7c078700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009870 tx=0x7ffb8402e000 comp rx=0 tx=0).stop 2024-05-11T01:13:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.039+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c071470 msgr2=0x7ffb9c155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.039+0000 7ffba3247640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c071470 0x7ffb9c155420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400c400 tx=0x7ffb9400c8c0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.039+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 shutdown_connections 2024-05-11T01:13:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.039+0000 7ffba3247640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ffb7c076240 0x7ffb7c078700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.039+0000 7ffba3247640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb9c15f420 0x7ffb9c0ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.039+0000 7ffba3247640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ffb9c0b2400 0x7ffb9c0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.039+0000 7ffba3247640 1 --2- 172.21.15.52:0/1337338977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c071470 0x7ffb9c155420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.040+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 >> 172.21.15.52:0/1337338977 conn(0x7ffb9c0d3080 msgr2=0x7ffb9c0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.040+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 shutdown_connections 2024-05-11T01:13:51.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.040+0000 7ffba3247640 1 -- 172.21.15.52:0/1337338977 wait complete. 2024-05-11T01:13:51.054 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2024-05-11T01:13:51.054 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2024-05-11T01:13:51.054 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-05-11T01:13:51.055 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:13:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.167+0000 7f114dd6a640 1 Processor -- start 2024-05-11T01:13:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.167+0000 7f114dd6a640 1 -- start start 2024-05-11T01:13:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f114dd6a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1148159980 0x7f1148162ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f114dd6a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f114806cb90 0x7f1148163420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f114dd6a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11480b6f90 con 0x7f114806fbc0 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f114dd6a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f1148159d60 con 0x7f114806cb90 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f114dd6a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f1148163d30 con 0x7f1148159980 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f1147fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f114806cb90 0x7f1148163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f1147fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f114806cb90 0x7f1148163420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36430/0 (socket says 172.21.15.52:36430) 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f1147fff640 1 -- 172.21.15.52:0/865421181 learned_addr learned my addr 172.21.15.52:0/865421181 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f11477fe640 1 --2- 172.21.15.52:0/865421181 >> v2:172.21.15.119:3300/0 conn(0x7f1148159980 0x7f1148162ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f1147fff640 1 -- 172.21.15.52:0/865421181 >> v2:172.21.15.119:3300/0 conn(0x7f1148159980 msgr2=0x7f1148162ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f1147fff640 1 --2- 172.21.15.52:0/865421181 >> v2:172.21.15.119:3300/0 conn(0x7f1148159980 0x7f1148162ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f1147fff640 1 -- 172.21.15.52:0/865421181 >> v1:172.21.15.119:6789/0 conn(0x7f114806fbc0 legacy=0x7f114806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.168+0000 7f1147fff640 1 -- 172.21.15.52:0/865421181 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1148163a50 con 0x7f114806cb90 2024-05-11T01:13:51.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.169+0000 7f1147fff640 1 --2- 172.21.15.52:0/865421181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 0x7f1148163420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f11480b4a80 tx=0x7f114002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d9231bb9bd8b3c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.169+0000 7f11467fc640 1 -- 172.21.15.52:0/865421181 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140036070 con 0x7f114806cb90 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.169+0000 7f11467fc640 1 -- 172.21.15.52:0/865421181 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1140031600 con 0x7f114806cb90 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.169+0000 7f11467fc640 1 -- 172.21.15.52:0/865421181 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11400319d0 con 0x7f114806cb90 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.170+0000 7f114dd6a640 1 -- 172.21.15.52:0/865421181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 msgr2=0x7f1148163420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.170+0000 7f114dd6a640 1 --2- 172.21.15.52:0/865421181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 0x7f1148163420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f11480b4a80 tx=0x7f114002a6a0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.170+0000 7f114dd6a640 1 -- 172.21.15.52:0/865421181 shutdown_connections 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.170+0000 7f114dd6a640 1 --2- 172.21.15.52:0/865421181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 0x7f1148163420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.170+0000 7f114dd6a640 1 --2- 172.21.15.52:0/865421181 >> v2:172.21.15.119:3300/0 conn(0x7f1148159980 0x7f1148162ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.170+0000 7f114dd6a640 1 -- 172.21.15.52:0/865421181 >> 172.21.15.52:0/865421181 conn(0x7f11480b3af0 msgr2=0x7f11480b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.170+0000 7f114dd6a640 1 -- 172.21.15.52:0/865421181 shutdown_connections 2024-05-11T01:13:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.170+0000 7f114dd6a640 1 -- 172.21.15.52:0/865421181 wait complete. 2024-05-11T01:13:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.171+0000 7f114dd6a640 1 Processor -- start 2024-05-11T01:13:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.171+0000 7f114dd6a640 1 -- start start 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f114dd6a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 0x7f11481446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f114dd6a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114806fbc0 0x7f11480b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f114dd6a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1148159980 0x7f1148143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f114dd6a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f114806f6a0 con 0x7f114806cb90 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f114dd6a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f114806f3c0 con 0x7f114806fbc0 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f114dd6a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f114815f8a0 con 0x7f1148159980 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f11477fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 0x7f11481446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f11477fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 0x7f11481446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36440/0 (socket says 172.21.15.52:36440) 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f11477fe640 1 -- 172.21.15.52:0/2878928500 learned_addr learned my addr 172.21.15.52:0/2878928500 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:51.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f1147fff640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1148159980 0x7f1148143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f1146ffd640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114806fbc0 0x7f11480b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f11477fe640 1 -- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1148159980 msgr2=0x7f1148143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f11477fe640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1148159980 0x7f1148143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f11477fe640 1 -- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114806fbc0 msgr2=0x7f11480b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f11477fe640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114806fbc0 0x7f11480b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f11477fe640 1 -- 172.21.15.52:0/2878928500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11481f4ce0 con 0x7f114806cb90 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f1146ffd640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114806fbc0 0x7f11480b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f11477fe640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 0x7f11481446b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f113800c7f0 tx=0x7f113800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.172+0000 7f112ffff640 1 -- 172.21.15.52:0/2878928500 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138015070 con 0x7f114806cb90 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.173+0000 7f112ffff640 1 -- 172.21.15.52:0/2878928500 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1138004220 con 0x7f114806cb90 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.173+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11480af070 con 0x7f114806cb90 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.173+0000 7f112ffff640 1 -- 172.21.15.52:0/2878928500 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138007620 con 0x7f114806cb90 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.173+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11480af630 con 0x7f114806cb90 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.174+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1148069780 con 0x7f114806cb90 2024-05-11T01:13:51.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.174+0000 7f112ffff640 1 -- 172.21.15.52:0/2878928500 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f11380079d0 con 0x7f114806cb90 2024-05-11T01:13:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.175+0000 7f112ffff640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f11340762d0 0x7f1134078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.175+0000 7f112ffff640 1 -- 172.21.15.52:0/2878928500 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7f1138097a30 con 0x7f114806cb90 2024-05-11T01:13:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.175+0000 7f1146ffd640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f11340762d0 0x7f1134078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.176+0000 7f1146ffd640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f11340762d0 0x7f1134078790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f11481f8370 tx=0x7f113c009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:51.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.179+0000 7f112ffff640 1 -- 172.21.15.52:0/2878928500 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1138060b90 con 0x7f114806cb90 2024-05-11T01:13:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.372+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1148076850 con 0x7f114806cb90 2024-05-11T01:13:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.373+0000 7f112ffff640 1 -- 172.21.15.52:0/2878928500 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+14814 (secure 0 0 0) 0x7f11380691d0 con 0x7f114806cb90 2024-05-11T01:13:51.373 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:51.373 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":52,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:13:49.028807+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/400712631":"2024-05-12T01:13:45.686878+0000","172.21.15.119:0/1003654821":"2024-05-12T01:13:45.686878+0000","172.21.15.119:0/884601481":"2024-05-12T01:13:45.686878+0000","172.21.15.52:6832/1325827549":"2024-05-12T01:13:45.397798+0000","172.21.15.119:6832/3083030853":"2024-05-12T01:01:57.869350+0000","172.21.15.119:0/1596302380":"2024-05-12T01:13:45.686878+0000","172.21.15.119:0/1042890889":"2024-05-12T01:01:57.869350+0000","172.21.15.119:6833/3083030853":"2024-05-12T01:01:57.869350+0000","172.21.15.119:0/221404270":"2024-05-12T01:13:45.686878+0000","172.21.15.119:0/1584246087":"2024-05-12T01:01:57.869350+0000","172.21.15.119:0/1820176553":"2024-05-12T01:01:57.869350+0000","172.21.15.52:6834/1325827549":"2024-05-12T01:13:45.397798+0000","172.21.15.119:0/224299274":"2024-05-12T01:13:45.686878+0000","172.21.15.119:6833/2051242002":"2024-05-12T01:13:45.686878+0000","172.21.15.119:0/2415667473":"2024-05-12T01:13:45.686878+0000","172.21.15.119:0/1530920757":"2024-05-12T01:01:57.869350+0000","172.21.15.119:6832/2051242002":"2024-05-12T01:13:45.686878+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":9,"length":1},{"begin":13,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":46,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":57,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":67,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":77,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":94,"length":2},{"begin":99,"length":1},{"begin":103,"length":1},{"begin":106,"length":1},{"begin":110,"length":1},{"begin":112,"length":1},{"begin":114,"length":1},{"begin":194,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f11340762d0 msgr2=0x7f1134078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f11340762d0 0x7f1134078790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f11481f8370 tx=0x7f113c009040 comp rx=0 tx=0).stop 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 msgr2=0x7f11481446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 0x7f11481446b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f113800c7f0 tx=0x7f113800ccb0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 shutdown_connections 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f11340762d0 0x7f1134078790 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1148159980 0x7f1148143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114806fbc0 0x7f11480b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 --2- 172.21.15.52:0/2878928500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f114806cb90 0x7f11481446b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 >> 172.21.15.52:0/2878928500 conn(0x7f11480b3af0 msgr2=0x7f11480732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 shutdown_connections 2024-05-11T01:13:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.375+0000 7f114dd6a640 1 -- 172.21.15.52:0/2878928500 wait complete. 2024-05-11T01:13:51.386 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T01:00:55.525012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T01:13:51.387 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T01:13:51.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.493+0000 7f32a9e45640 1 Processor -- start 2024-05-11T01:13:51.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.494+0000 7f32a9e45640 1 -- start start 2024-05-11T01:13:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.494+0000 7f32a9e45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 0x7f32a40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.494+0000 7f32a9e45640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f32a40d2d60 0x7f32a40d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.494+0000 7f32a9e45640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a40d3a90 con 0x7f32a40d4e40 2024-05-11T01:13:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.494+0000 7f32a9e45640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f32a40d3d70 con 0x7f32a40d2d60 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.494+0000 7f32a9e45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f32a40d4050 con 0x7f32a415f2f0 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.494+0000 7f32a2ffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f32a40d4e40 0x7f32a40d5280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:36408/0 (socket says 172.21.15.52:36408) 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.495+0000 7f32a2ffd640 1 -- 172.21.15.52:0/3893791583 learned_addr learned my addr 172.21.15.52:0/3893791583 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.495+0000 7f32a37fe640 1 --2- 172.21.15.52:0/3893791583 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 0x7f32a40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.495+0000 7f32a3fff640 1 --2- 172.21.15.52:0/3893791583 >> v2:172.21.15.119:3300/0 conn(0x7f32a40d2d60 0x7f32a40d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.495+0000 7f32a27fc640 1 -- 172.21.15.52:0/3893791583 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2304928089 0 0) 0x7f32a40d3a90 con 0x7f32a40d4e40 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.495+0000 7f32a27fc640 1 -- 172.21.15.52:0/3893791583 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f328c003520 con 0x7f32a40d4e40 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.495+0000 7f32a37fe640 1 -- 172.21.15.52:0/3893791583 >> v2:172.21.15.119:3300/0 conn(0x7f32a40d2d60 msgr2=0x7f32a40d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.495+0000 7f32a37fe640 1 --2- 172.21.15.52:0/3893791583 >> v2:172.21.15.119:3300/0 conn(0x7f32a40d2d60 0x7f32a40d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.495+0000 7f32a37fe640 1 -- 172.21.15.52:0/3893791583 >> v1:172.21.15.52:6789/0 conn(0x7f32a40d4e40 legacy=0x7f32a40d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.495+0000 7f32a37fe640 1 -- 172.21.15.52:0/3893791583 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a40b4e60 con 0x7f32a415f2f0 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.496+0000 7f32a37fe640 1 --2- 172.21.15.52:0/3893791583 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 0x7f32a40cfca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32940097a0 tx=0x7f329402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f11d5e19b9be172 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.496+0000 7f32a27fc640 1 -- 172.21.15.52:0/3893791583 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294037070 con 0x7f32a415f2f0 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.496+0000 7f32a27fc640 1 -- 172.21.15.52:0/3893791583 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3294032440 con 0x7f32a415f2f0 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.496+0000 7f32a27fc640 1 -- 172.21.15.52:0/3893791583 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294032810 con 0x7f32a415f2f0 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.497+0000 7f32a9e45640 1 -- 172.21.15.52:0/3893791583 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 msgr2=0x7f32a40cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.497+0000 7f32a9e45640 1 --2- 172.21.15.52:0/3893791583 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 0x7f32a40cfca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32940097a0 tx=0x7f329402a180 comp rx=0 tx=0).stop 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.497+0000 7f32a9e45640 1 -- 172.21.15.52:0/3893791583 shutdown_connections 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.497+0000 7f32a9e45640 1 --2- 172.21.15.52:0/3893791583 >> v2:172.21.15.119:3300/0 conn(0x7f32a40d2d60 0x7f32a40d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.497+0000 7f32a9e45640 1 --2- 172.21.15.52:0/3893791583 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 0x7f32a40cfca0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.497+0000 7f32a9e45640 1 -- 172.21.15.52:0/3893791583 >> 172.21.15.52:0/3893791583 conn(0x7f32a408ebb0 msgr2=0x7f32a409e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.497+0000 7f32a9e45640 1 -- 172.21.15.52:0/3893791583 shutdown_connections 2024-05-11T01:13:51.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.497+0000 7f32a9e45640 1 -- 172.21.15.52:0/3893791583 wait complete. 2024-05-11T01:13:51.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.498+0000 7f32a9e45640 1 Processor -- start 2024-05-11T01:13:51.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.498+0000 7f32a9e45640 1 -- start start 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a9e45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a40d2d60 0x7f32a40c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a9e45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40d4e40 0x7f32a40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a9e45640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 0x7f32a40cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a9e45640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32a40d01e0 con 0x7f32a40d4e40 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a9e45640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32a40d3710 con 0x7f32a40d2d60 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a9e45640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f32a40d0360 con 0x7f32a415f2f0 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40d4e40 0x7f32a40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40d4e40 0x7f32a40cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36456/0 (socket says 172.21.15.52:36456) 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a2ffd640 1 -- 172.21.15.52:0/3458741848 learned_addr learned my addr 172.21.15.52:0/3458741848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:51.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a3fff640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 0x7f32a40cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a37fe640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a40d2d60 0x7f32a40c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a2ffd640 1 -- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 msgr2=0x7f32a40cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a2ffd640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 0x7f32a40cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a2ffd640 1 -- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a40d2d60 msgr2=0x7f32a40c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a2ffd640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a40d2d60 0x7f32a40c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a2ffd640 1 -- 172.21.15.52:0/3458741848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a40ca250 con 0x7f32a40d4e40 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a2ffd640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40d4e40 0x7f32a40cecf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f329800c530 tx=0x7f329800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f328bfff640 1 -- 172.21.15.52:0/3458741848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298017070 con 0x7f32a40d4e40 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f328bfff640 1 -- 172.21.15.52:0/3458741848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3298005ce0 con 0x7f32a40d4e40 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.499+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a40ca520 con 0x7f32a40d4e40 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.500+0000 7f328bfff640 1 -- 172.21.15.52:0/3458741848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298007270 con 0x7f32a40d4e40 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.500+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a40caae0 con 0x7f32a40d4e40 2024-05-11T01:13:51.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.500+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a40695d0 con 0x7f32a40d4e40 2024-05-11T01:13:51.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.501+0000 7f328bfff640 1 -- 172.21.15.52:0/3458741848 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f329801a070 con 0x7f32a40d4e40 2024-05-11T01:13:51.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.502+0000 7f328bfff640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32900762d0 0x7f3290078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.502+0000 7f328bfff640 1 -- 172.21.15.52:0/3458741848 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7f3298096960 con 0x7f32a40d4e40 2024-05-11T01:13:51.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.502+0000 7f32a37fe640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32900762d0 0x7f3290078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.502+0000 7f32a37fe640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32900762d0 0x7f3290078790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3294034000 tx=0x7f329402e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:51.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.506+0000 7f328bfff640 1 -- 172.21.15.52:0/3458741848 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f329805fac0 con 0x7f32a40d4e40 2024-05-11T01:13:51.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.695+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32a40d6080 con 0x7f32a40d4e40 2024-05-11T01:13:51.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.695+0000 7f328bfff640 1 -- 172.21.15.52:0/3458741848 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f3298063dc0 con 0x7f32a40d4e40 2024-05-11T01:13:51.695 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-05-11T01:13:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32900762d0 msgr2=0x7f3290078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32900762d0 0x7f3290078790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3294034000 tx=0x7f329402e000 comp rx=0 tx=0).stop 2024-05-11T01:13:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40d4e40 msgr2=0x7f32a40cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40d4e40 0x7f32a40cecf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f329800c530 tx=0x7f329800c9f0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 shutdown_connections 2024-05-11T01:13:51.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32900762d0 0x7f3290078790 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f32a415f2f0 0x7f32a40cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32a40d4e40 0x7f32a40cecf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 --2- 172.21.15.52:0/3458741848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a40d2d60 0x7f32a40c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 >> 172.21.15.52:0/3458741848 conn(0x7f32a408ebb0 msgr2=0x7f32a40b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:51.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 shutdown_connections 2024-05-11T01:13:51.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.697+0000 7f32a9e45640 1 -- 172.21.15.52:0/3458741848 wait complete. 2024-05-11T01:13:51.709 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2024-05-11T01:13:51.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.813+0000 7f696d3b7640 1 Processor -- start 2024-05-11T01:13:51.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.813+0000 7f696d3b7640 1 -- start start 2024-05-11T01:13:51.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.814+0000 7f696d3b7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f696815c3f0 0x7f69680d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.814+0000 7f696d3b7640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6968073250 0x7f69680736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.814+0000 7f696d3b7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f696815fe50 con 0x7f69680717a0 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.814+0000 7f696d3b7640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f6968160150 con 0x7f6968073250 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.814+0000 7f696d3b7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f6968160cf0 con 0x7f696815c3f0 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.814+0000 7f69677fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6968073250 0x7f69680736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.815+0000 7f69677fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6968073250 0x7f69680736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59522/0 (socket says 172.21.15.52:59522) 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.815+0000 7f69677fe640 1 -- 172.21.15.52:0/939641766 learned_addr learned my addr 172.21.15.52:0/939641766 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.815+0000 7f6966ffd640 1 --2- 172.21.15.52:0/939641766 >> v2:172.21.15.119:3300/0 conn(0x7f696815c3f0 0x7f69680d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.815+0000 7f69677fe640 1 -- 172.21.15.52:0/939641766 >> v2:172.21.15.119:3300/0 conn(0x7f696815c3f0 msgr2=0x7f69680d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.815+0000 7f69677fe640 1 --2- 172.21.15.52:0/939641766 >> v2:172.21.15.119:3300/0 conn(0x7f696815c3f0 0x7f69680d0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.815+0000 7f69677fe640 1 -- 172.21.15.52:0/939641766 >> v1:172.21.15.119:6789/0 conn(0x7f69680717a0 legacy=0x7f69681605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.815+0000 7f69677fe640 1 -- 172.21.15.52:0/939641766 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6968071c20 con 0x7f6968073250 2024-05-11T01:13:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.815+0000 7f69677fe640 1 --2- 172.21.15.52:0/939641766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 0x7f69680736e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6960009870 tx=0x7f696002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadfc4ecc677a482 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f6965ffb640 1 -- 172.21.15.52:0/939641766 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960037070 con 0x7f6968073250 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f6965ffb640 1 -- 172.21.15.52:0/939641766 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6960032440 con 0x7f6968073250 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f6965ffb640 1 -- 172.21.15.52:0/939641766 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960032810 con 0x7f6968073250 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f696d3b7640 1 -- 172.21.15.52:0/939641766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 msgr2=0x7f69680736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f696d3b7640 1 --2- 172.21.15.52:0/939641766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 0x7f69680736e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6960009870 tx=0x7f696002a210 comp rx=0 tx=0).stop 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f696d3b7640 1 -- 172.21.15.52:0/939641766 shutdown_connections 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f696d3b7640 1 --2- 172.21.15.52:0/939641766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 0x7f69680736e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f696d3b7640 1 --2- 172.21.15.52:0/939641766 >> v2:172.21.15.119:3300/0 conn(0x7f696815c3f0 0x7f69680d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f696d3b7640 1 -- 172.21.15.52:0/939641766 >> 172.21.15.52:0/939641766 conn(0x7f69680b3a30 msgr2=0x7f69680b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f696d3b7640 1 -- 172.21.15.52:0/939641766 shutdown_connections 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.816+0000 7f696d3b7640 1 -- 172.21.15.52:0/939641766 wait complete. 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.818+0000 7f696d3b7640 1 Processor -- start 2024-05-11T01:13:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.818+0000 7f696d3b7640 1 -- start start 2024-05-11T01:13:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.818+0000 7f696d3b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69680717a0 0x7f6968152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.818+0000 7f696d3b7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 0x7f696815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.818+0000 7f696d3b7640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f696815c3f0 0x7f69680b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.818+0000 7f696d3b7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69680d1b30 con 0x7f6968073250 2024-05-11T01:13:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.818+0000 7f696d3b7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69680d1850 con 0x7f69680717a0 2024-05-11T01:13:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.818+0000 7f696d3b7640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f696815c7d0 con 0x7f696815c3f0 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69667fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 0x7f696815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69667fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 0x7f696815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59526/0 (socket says 172.21.15.52:59526) 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69667fc640 1 -- 172.21.15.52:0/3796462279 learned_addr learned my addr 172.21.15.52:0/3796462279 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69677fe640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f696815c3f0 0x7f69680b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f6966ffd640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69680717a0 0x7f6968152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69667fc640 1 -- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f696815c3f0 msgr2=0x7f69680b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69667fc640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f696815c3f0 0x7f69680b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69667fc640 1 -- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69680717a0 msgr2=0x7f6968152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69667fc640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69680717a0 0x7f6968152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69667fc640 1 -- 172.21.15.52:0/3796462279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69680c81d0 con 0x7f6968073250 2024-05-11T01:13:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f6966ffd640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69680717a0 0x7f6968152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f69667fc640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 0x7f696815a680 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f695000c460 tx=0x7f695000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f695f7fe640 1 -- 172.21.15.52:0/3796462279 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950016020 con 0x7f6968073250 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f695f7fe640 1 -- 172.21.15.52:0/3796462279 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6950012870 con 0x7f6968073250 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69680c8470 con 0x7f6968073250 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f695f7fe640 1 -- 172.21.15.52:0/3796462279 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950012c60 con 0x7f6968073250 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.819+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69680c8a30 con 0x7f6968073250 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.820+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6968069670 con 0x7f6968073250 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.821+0000 7f695f7fe640 1 -- 172.21.15.52:0/3796462279 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f6950019020 con 0x7f6968073250 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.822+0000 7f695f7fe640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f69480762d0 0x7f6948078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.822+0000 7f695f7fe640 1 -- 172.21.15.52:0/3796462279 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7f6950097150 con 0x7f6968073250 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.822+0000 7f6966ffd640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f69480762d0 0x7f6948078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.822+0000 7f6966ffd640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f69480762d0 0x7f6948078790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6958007fc0 tx=0x7f69580075b0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:51.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:51.826+0000 7f695f7fe640 1 -- 172.21.15.52:0/3796462279 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6950020070 con 0x7f6968073250 2024-05-11T01:13:52.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:52.101+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) -- 0x7f69680cd5a0 con 0x7f6968073250 2024-05-11T01:13:53.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.044+0000 7f695f7fe640 1 -- 172.21.15.52:0/3796462279 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v445) ==== 157+0+0 (secure 0 0 0) 0x7f6950060900 con 0x7f6968073250 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.047+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f69480762d0 msgr2=0x7f6948078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.047+0000 7f696d3b7640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f69480762d0 0x7f6948078790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6958007fc0 tx=0x7f69580075b0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.047+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 msgr2=0x7f696815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.047+0000 7f696d3b7640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 0x7f696815a680 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f695000c460 tx=0x7f695000c920 comp rx=0 tx=0).stop 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.047+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 shutdown_connections 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.048+0000 7f696d3b7640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f69480762d0 0x7f6948078790 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.048+0000 7f696d3b7640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f696815c3f0 0x7f69680b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.048+0000 7f696d3b7640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6968073250 0x7f696815a680 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.048+0000 7f696d3b7640 1 --2- 172.21.15.52:0/3796462279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69680717a0 0x7f6968152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.048+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 >> 172.21.15.52:0/3796462279 conn(0x7f69680b3a30 msgr2=0x7f69680b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:53.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.048+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 shutdown_connections 2024-05-11T01:13:53.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.048+0000 7f696d3b7640 1 -- 172.21.15.52:0/3796462279 wait complete. 2024-05-11T01:13:53.061 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-11T01:13:53.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f6051247640 1 Processor -- start 2024-05-11T01:13:53.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f6051247640 1 -- start start 2024-05-11T01:13:53.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f6051247640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c15f430 0x7f604c0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f6051247640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f604c0d0e80 0x7f604c0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f6051247640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f604c1632b0 con 0x7f604c0d1a30 2024-05-11T01:13:53.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f6051247640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f604c15f810 con 0x7f604c0d0e80 2024-05-11T01:13:53.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f6051247640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f604c163590 con 0x7f604c15f430 2024-05-11T01:13:53.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f604b59e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f604c0d0e80 0x7f604c0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f604b59e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f604c0d0e80 0x7f604c0cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59540/0 (socket says 172.21.15.52:59540) 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f604b59e640 1 -- 172.21.15.52:0/312376089 learned_addr learned my addr 172.21.15.52:0/312376089 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f604ad9d640 1 --2- 172.21.15.52:0/312376089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c15f430 0x7f604c0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f604b59e640 1 -- 172.21.15.52:0/312376089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c15f430 msgr2=0x7f604c0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f604b59e640 1 --2- 172.21.15.52:0/312376089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c15f430 0x7f604c0cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f604b59e640 1 -- 172.21.15.52:0/312376089 >> v1:172.21.15.119:6789/0 conn(0x7f604c0d1a30 legacy=0x7f604c073270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f604b59e640 1 -- 172.21.15.52:0/312376089 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c163870 con 0x7f604c0d0e80 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.176+0000 7f604ad9d640 1 --2- 172.21.15.52:0/312376089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c15f430 0x7f604c0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.177+0000 7f604b59e640 1 --2- 172.21.15.52:0/312376089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d0e80 0x7f604c0cca50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6044009e50 tx=0x7f604402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a187fbe149b42e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.177+0000 7f604a59c640 1 -- 172.21.15.52:0/312376089 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044036070 con 0x7f604c0d0e80 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.177+0000 7f604a59c640 1 -- 172.21.15.52:0/312376089 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6044031600 con 0x7f604c0d0e80 2024-05-11T01:13:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.177+0000 7f604a59c640 1 -- 172.21.15.52:0/312376089 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440319d0 con 0x7f604c0d0e80 2024-05-11T01:13:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.178+0000 7f6051247640 1 -- 172.21.15.52:0/312376089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d0e80 msgr2=0x7f604c0cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.178+0000 7f6051247640 1 --2- 172.21.15.52:0/312376089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d0e80 0x7f604c0cca50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6044009e50 tx=0x7f604402a5f0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.178+0000 7f6051247640 1 -- 172.21.15.52:0/312376089 shutdown_connections 2024-05-11T01:13:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.178+0000 7f6051247640 1 --2- 172.21.15.52:0/312376089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d0e80 0x7f604c0cca50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.178+0000 7f6051247640 1 --2- 172.21.15.52:0/312376089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c15f430 0x7f604c0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.178+0000 7f6051247640 1 -- 172.21.15.52:0/312376089 >> 172.21.15.52:0/312376089 conn(0x7f604c0ce940 msgr2=0x7f604c0ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.178+0000 7f6051247640 1 -- 172.21.15.52:0/312376089 shutdown_connections 2024-05-11T01:13:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.178+0000 7f6051247640 1 -- 172.21.15.52:0/312376089 wait complete. 2024-05-11T01:13:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.179+0000 7f6051247640 1 Processor -- start 2024-05-11T01:13:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.179+0000 7f6051247640 1 -- start start 2024-05-11T01:13:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f6051247640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c0d0e80 0x7f604c15d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f6051247640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d1a30 0x7f604c0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f6051247640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f604c15f430 0x7f604c0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f6051247640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f604c163030 con 0x7f604c0d1a30 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f6051247640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f604c162eb0 con 0x7f604c15f430 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f6051247640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f604c071690 con 0x7f604c0d0e80 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f6043fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d1a30 0x7f604c0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f6043fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d1a30 0x7f604c0c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59550/0 (socket says 172.21.15.52:59550) 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f6043fff640 1 -- 172.21.15.52:0/3954027311 learned_addr learned my addr 172.21.15.52:0/3954027311 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f604ad9d640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c0d0e80 0x7f604c15d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.180+0000 7f604b59e640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f604c15f430 0x7f604c0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f6043fff640 1 -- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c0d0e80 msgr2=0x7f604c15d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f6043fff640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c0d0e80 0x7f604c15d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f6043fff640 1 -- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f604c15f430 msgr2=0x7f604c0c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f6043fff640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f604c15f430 0x7f604c0c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f6043fff640 1 -- 172.21.15.52:0/3954027311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c0b63b0 con 0x7f604c0d1a30 2024-05-11T01:13:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f604ad9d640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c0d0e80 0x7f604c15d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f6043fff640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d1a30 0x7f604c0c7cf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f603400c400 tx=0x7f603400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f60437fe640 1 -- 172.21.15.52:0/3954027311 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034011790 con 0x7f604c0d1a30 2024-05-11T01:13:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f60437fe640 1 -- 172.21.15.52:0/3954027311 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6034017070 con 0x7f604c0d1a30 2024-05-11T01:13:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c0c7210 con 0x7f604c0d1a30 2024-05-11T01:13:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f60437fe640 1 -- 172.21.15.52:0/3954027311 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034011bb0 con 0x7f604c0d1a30 2024-05-11T01:13:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.181+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c0c77d0 con 0x7f604c0d1a30 2024-05-11T01:13:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.182+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604c0695d0 con 0x7f604c0d1a30 2024-05-11T01:13:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.183+0000 7f60437fe640 1 -- 172.21.15.52:0/3954027311 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f603401c460 con 0x7f604c0d1a30 2024-05-11T01:13:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.183+0000 7f60437fe640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f602c0762d0 0x7f602c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.183+0000 7f60437fe640 1 -- 172.21.15.52:0/3954027311 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7f60340975a0 con 0x7f604c0d1a30 2024-05-11T01:13:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.183+0000 7f604ad9d640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f602c0762d0 0x7f602c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.184+0000 7f604ad9d640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f602c0762d0 0x7f602c078790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f603c00a990 tx=0x7f603c008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:53.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.187+0000 7f60437fe640 1 -- 172.21.15.52:0/3954027311 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6034060de0 con 0x7f604c0d1a30 2024-05-11T01:13:53.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.378+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f604c0c79a0 con 0x7f604c0d1a30 2024-05-11T01:13:53.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.382+0000 7f60437fe640 1 -- 172.21.15.52:0/3954027311 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f60340606e0 con 0x7f604c0d1a30 2024-05-11T01:13:53.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f602c0762d0 msgr2=0x7f602c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f602c0762d0 0x7f602c078790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f603c00a990 tx=0x7f603c008040 comp rx=0 tx=0).stop 2024-05-11T01:13:53.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d1a30 msgr2=0x7f604c0c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d1a30 0x7f604c0c7cf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f603400c400 tx=0x7f603400c8c0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 shutdown_connections 2024-05-11T01:13:53.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f602c0762d0 0x7f602c078790 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f604c15f430 0x7f604c0c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f604c0d1a30 0x7f604c0c7cf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 --2- 172.21.15.52:0/3954027311 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f604c0d0e80 0x7f604c15d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 >> 172.21.15.52:0/3954027311 conn(0x7f604c0ce940 msgr2=0x7f604c0afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:53.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 shutdown_connections 2024-05-11T01:13:53.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.384+0000 7f6051247640 1 -- 172.21.15.52:0/3954027311 wait complete. 2024-05-11T01:13:53.395 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T01:13:53.396 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T01:13:53.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T01:13:53.495 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T01:13:53.495 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:13:53.495 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:13:53.512 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3484 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T01:13:53.512 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:13:53.512 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:13:53.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13678 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T01:13:53.554 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:13:53.554 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:13:53.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3484 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T01:13:53.570 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:13:53.570 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:13:53.612 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13678 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T01:13:53.612 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-11T01:13:53.612 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:13:53.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.717+0000 7feb5b946640 1 Processor -- start 2024-05-11T01:13:53.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.717+0000 7feb5b946640 1 -- start start 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb5b946640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb5415f2d0 0x7feb5406e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb5b946640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb5406e5d0 0x7feb540d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb5b946640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feb540d33e0 0x7feb540cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb5b946640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7feb540d3810 con 0x7feb540d33e0 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb5b946640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7feb540d1f90 con 0x7feb5406e5d0 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb5b946640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7feb540d2270 con 0x7feb5415f2d0 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb59ee3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feb540d33e0 0x7feb540cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb59ee3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7feb540d33e0 0x7feb540cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59562/0 (socket says 172.21.15.52:59562) 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb59ee3640 1 -- 172.21.15.52:0/3623051671 learned_addr learned my addr 172.21.15.52:0/3623051671 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb596e2640 1 --2- 172.21.15.52:0/3623051671 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb5415f2d0 0x7feb5406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb58ee1640 1 --2- 172.21.15.52:0/3623051671 >> v2:172.21.15.119:3300/0 conn(0x7feb5406e5d0 0x7feb540d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb59ee3640 1 -- 172.21.15.52:0/3623051671 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb5415f2d0 msgr2=0x7feb5406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb59ee3640 1 --2- 172.21.15.52:0/3623051671 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb5415f2d0 0x7feb5406e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb59ee3640 1 -- 172.21.15.52:0/3623051671 >> v2:172.21.15.119:3300/0 conn(0x7feb5406e5d0 msgr2=0x7feb540d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb59ee3640 1 --2- 172.21.15.52:0/3623051671 >> v2:172.21.15.119:3300/0 conn(0x7feb5406e5d0 0x7feb540d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb59ee3640 1 -- 172.21.15.52:0/3623051671 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb540d2550 con 0x7feb540d33e0 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb59ee3640 1 --2- 172.21.15.52:0/3623051671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb540d33e0 0x7feb540cf7b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feb540747e0 tx=0x7feb4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1754a7cd13d57a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb4bfff640 1 -- 172.21.15.52:0/3623051671 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c037070 con 0x7feb540d33e0 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb4bfff640 1 -- 172.21.15.52:0/3623051671 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb4c032440 con 0x7feb540d33e0 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.718+0000 7feb4bfff640 1 -- 172.21.15.52:0/3623051671 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c032810 con 0x7feb540d33e0 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.719+0000 7feb5b946640 1 -- 172.21.15.52:0/3623051671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb540d33e0 msgr2=0x7feb540cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.719+0000 7feb5b946640 1 --2- 172.21.15.52:0/3623051671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb540d33e0 0x7feb540cf7b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feb540747e0 tx=0x7feb4c02a1e0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.719+0000 7feb5b946640 1 -- 172.21.15.52:0/3623051671 shutdown_connections 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.719+0000 7feb5b946640 1 --2- 172.21.15.52:0/3623051671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb540d33e0 0x7feb540cf7b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.719+0000 7feb5b946640 1 --2- 172.21.15.52:0/3623051671 >> v2:172.21.15.119:3300/0 conn(0x7feb5406e5d0 0x7feb540d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.719+0000 7feb5b946640 1 --2- 172.21.15.52:0/3623051671 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb5415f2d0 0x7feb5406e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.719+0000 7feb5b946640 1 -- 172.21.15.52:0/3623051671 >> 172.21.15.52:0/3623051671 conn(0x7feb5409ba20 msgr2=0x7feb5409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.719+0000 7feb5b946640 1 -- 172.21.15.52:0/3623051671 shutdown_connections 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.719+0000 7feb5b946640 1 -- 172.21.15.52:0/3623051671 wait complete. 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.720+0000 7feb5b946640 1 Processor -- start 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.720+0000 7feb5b946640 1 -- start start 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb5b946640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb5406e5d0 0x7feb541f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb5b946640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb540d33e0 0x7feb54155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb5b946640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb5415f2d0 0x7feb5414b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb5b946640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb5415f6b0 con 0x7feb5415f2d0 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb5b946640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb540d2b10 con 0x7feb5406e5d0 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb5b946640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7feb5415f830 con 0x7feb540d33e0 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb59ee3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb5415f2d0 0x7feb5414b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb59ee3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb5415f2d0 0x7feb5414b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59578/0 (socket says 172.21.15.52:59578) 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb59ee3640 1 -- 172.21.15.52:0/322908420 learned_addr learned my addr 172.21.15.52:0/322908420 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb58ee1640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb540d33e0 0x7feb54155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb596e2640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb5406e5d0 0x7feb541f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb59ee3640 1 -- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb540d33e0 msgr2=0x7feb54155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb59ee3640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb540d33e0 0x7feb54155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb59ee3640 1 -- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb5406e5d0 msgr2=0x7feb541f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb59ee3640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb5406e5d0 0x7feb541f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb59ee3640 1 -- 172.21.15.52:0/322908420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb54155590 con 0x7feb5415f2d0 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.721+0000 7feb59ee3640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb5415f2d0 0x7feb5414b2c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feb4c02a220 tx=0x7feb4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.722+0000 7feb49ffb640 1 -- 172.21.15.52:0/322908420 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c037040 con 0x7feb5415f2d0 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.722+0000 7feb49ffb640 1 -- 172.21.15.52:0/322908420 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb4c042070 con 0x7feb5415f2d0 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.722+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb54155840 con 0x7feb5415f2d0 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.722+0000 7feb49ffb640 1 -- 172.21.15.52:0/322908420 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c030920 con 0x7feb5415f2d0 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.722+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb54155e00 con 0x7feb5415f2d0 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.723+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb54156000 con 0x7feb5415f2d0 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.723+0000 7feb49ffb640 1 -- 172.21.15.52:0/322908420 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7feb4c030ac0 con 0x7feb5415f2d0 2024-05-11T01:13:53.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.724+0000 7feb49ffb640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7feb380762d0 0x7feb38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:53.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.724+0000 7feb49ffb640 1 -- 172.21.15.52:0/322908420 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7feb4c0b7f50 con 0x7feb5415f2d0 2024-05-11T01:13:53.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.724+0000 7feb596e2640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7feb380762d0 0x7feb38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:53.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.726+0000 7feb596e2640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7feb380762d0 0x7feb38078790 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feb50003aa0 tx=0x7feb500098d0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:53.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.729+0000 7feb49ffb640 1 -- 172.21.15.52:0/322908420 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feb4c0810b0 con 0x7feb5415f2d0 2024-05-11T01:13:53.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.960+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb540afdf0 con 0x7feb5415f2d0 2024-05-11T01:13:53.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.960+0000 7feb49ffb640 1 -- 172.21.15.52:0/322908420 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3869 (secure 0 0 0) 0x7feb4c0853b0 con 0x7feb5415f2d0 2024-05-11T01:13:53.960 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:53.960 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3122737368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3122737368},{"type":"v1","addr":"172.21.15.119:6836","nonce":3122737368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/211602681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":211602681},{"type":"v1","addr":"172.21.15.119:6837","nonce":211602681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2079689543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2079689543},{"type":"v1","addr":"172.21.15.52:6835","nonce":2079689543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5060,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/312548550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":312548550},{"type":"v1","addr":"172.21.15.52:6834","nonce":312548550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-05-11T01:13:53.961 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 19 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7feb380762d0 msgr2=0x7feb38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7feb380762d0 0x7feb38078790 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feb50003aa0 tx=0x7feb500098d0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb5415f2d0 msgr2=0x7feb5414b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb5415f2d0 0x7feb5414b2c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feb4c02a220 tx=0x7feb4c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 shutdown_connections 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7feb380762d0 0x7feb38078790 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb5415f2d0 0x7feb5414b2c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feb540d33e0 0x7feb54155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 --2- 172.21.15.52:0/322908420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb5406e5d0 0x7feb541f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 >> 172.21.15.52:0/322908420 conn(0x7feb5409ba20 msgr2=0x7feb540b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:53.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 shutdown_connections 2024-05-11T01:13:53.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:53.963+0000 7feb5b946640 1 -- 172.21.15.52:0/322908420 wait complete. 2024-05-11T01:13:53.974 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T01:13:53.974 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-11T01:13:53.974 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T01:13:53.974 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:13:53.975 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T01:13:53.975 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-11T01:13:53.975 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T01:13:53.975 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:13:53.975 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T01:13:53.975 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-11T01:13:53.975 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T01:13:53.976 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:13:53.976 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T01:13:53.976 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-11T01:13:53.976 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T01:13:53.976 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:13:53.984 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T01:13:53.984 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T01:13:53.985 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T01:13:53.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T01:13:53.988 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T01:13:53.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T01:13:53.988 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T01:13:53.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T01:13:54.027 INFO:tasks.ceph.mds.a:Started 2024-05-11T01:13:54.027 DEBUG:teuthology.parallel:result is None 2024-05-11T01:13:54.028 INFO:tasks.ceph.mds.c:Started 2024-05-11T01:13:54.028 DEBUG:teuthology.parallel:result is None 2024-05-11T01:13:54.030 INFO:tasks.ceph.mds.d:Started 2024-05-11T01:13:54.030 DEBUG:teuthology.parallel:result is None 2024-05-11T01:13:54.031 INFO:tasks.ceph.mds.b:Started 2024-05-11T01:13:54.031 DEBUG:teuthology.parallel:result is None 2024-05-11T01:13:54.031 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-11T01:13:54.107 INFO:tasks.ceph.mds.c.smithi052.stdout:starting mds.c at 2024-05-11T01:13:54.107 INFO:tasks.ceph.mds.a.smithi052.stdout:starting mds.a at 2024-05-11T01:13:54.112 INFO:tasks.ceph.mds.d.smithi119.stdout:starting mds.d at 2024-05-11T01:13:54.112 INFO:tasks.ceph.mds.b.smithi119.stdout:starting mds.b at 2024-05-11T01:13:54.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.141+0000 7f56d4a12640 1 Processor -- start 2024-05-11T01:13:54.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.141+0000 7f56d4a12640 1 -- start start 2024-05-11T01:13:54.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.142+0000 7f56d4a12640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d0160480 0x7f56d00d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:54.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.142+0000 7f56d4a12640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d0163250 con 0x7f56d00d32c0 2024-05-11T01:13:54.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.142+0000 7f56d4a12640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d0163530 con 0x7f56d015f430 2024-05-11T01:13:54.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.142+0000 7f56d4a12640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f56d0163810 con 0x7f56d0160480 2024-05-11T01:13:54.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.142+0000 7f56ced9d640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f56d00d32c0 0x7f56d00d3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:47956/0 (socket says 172.21.15.52:47956) 2024-05-11T01:13:54.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.142+0000 7f56ced9d640 1 -- 172.21.15.52:0/3098174816 learned_addr learned my addr 172.21.15.52:0/3098174816 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.142+0000 7f56cdd9b640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d0160480 0x7f56d00d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.142+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 216909173 0 0) 0x7f56d0163250 con 0x7f56d00d32c0 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.142+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56b8003520 con 0x7f56d00d32c0 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 207392746 0 0) 0x7f56b8003520 con 0x7f56d00d32c0 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d0160480 msgr2=0x7f56d00d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d0160480 0x7f56d00d2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 >> v1:172.21.15.119:6789/0 conn(0x7f56d015f430 legacy=0x7f56d015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d0163af0 con 0x7f56d00d32c0 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f56c8001a70 con 0x7f56d00d32c0 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 >> v1:172.21.15.52:6789/0 conn(0x7f56d00d32c0 legacy=0x7f56d00d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56b8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d0160480 0x7f56b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b8004d40 0x7f56b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0163810 con 0x7f56b8003a30 2024-05-11T01:13:54.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56b800bb00 con 0x7f56b8004d40 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f56b800bd00 con 0x7f56d0160480 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56ce59c640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56b8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56cdd9b640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d0160480 0x7f56b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56ced9d640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b8004d40 0x7f56b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56ce59c640 1 -- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d0160480 msgr2=0x7f56b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56ce59c640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d0160480 0x7f56b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56ce59c640 1 -- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b8004d40 msgr2=0x7f56b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56ce59c640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b8004d40 0x7f56b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.143+0000 7f56ce59c640 1 -- 172.21.15.52:0/3098174816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b8003520 con 0x7f56b8003a30 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.144+0000 7f56ce59c640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56b8003e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f56c0001810 tx=0x7f56b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400202f6881a4702 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.144+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0037070 con 0x7f56b8003a30 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.144+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c002aba0 con 0x7f56b8003a30 2024-05-11T01:13:54.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.144+0000 7f56cd59a640 1 -- 172.21.15.52:0/3098174816 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0001430 con 0x7f56b8003a30 2024-05-11T01:13:54.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.145+0000 7f56d4a12640 1 -- 172.21.15.52:0/3098174816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 msgr2=0x7f56b8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:54.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.145+0000 7f56d4a12640 1 --2- 172.21.15.52:0/3098174816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56b8003e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f56c0001810 tx=0x7f56b8005830 comp rx=0 tx=0).stop 2024-05-11T01:13:54.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.145+0000 7f56ced9d640 1 -- 172.21.15.52:0/3098174816 reap_dead start 2024-05-11T01:13:54.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.145+0000 7f56d4a12640 1 -- 172.21.15.52:0/3098174816 shutdown_connections 2024-05-11T01:13:54.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.145+0000 7f56d4a12640 1 -- 172.21.15.52:0/3098174816 >> 172.21.15.52:0/3098174816 conn(0x7f56d008eb70 msgr2=0x7f56d009e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:54.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.145+0000 7f56d4a12640 1 -- 172.21.15.52:0/3098174816 shutdown_connections 2024-05-11T01:13:54.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.145+0000 7f56d4a12640 1 -- 172.21.15.52:0/3098174816 wait complete. 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.146+0000 7f56d4a12640 1 Processor -- start 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.146+0000 7f56d4a12640 1 -- start start 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56d4a12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56d00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56d4a12640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d00b30b0 0x7f56d01510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56d4a12640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d00b3530 0x7f56d0151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56d4a12640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0163030 con 0x7f56b8003a30 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56d4a12640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0162eb0 con 0x7f56d00b30b0 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56d4a12640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f56d01608c0 con 0x7f56d00b3530 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ce59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56d00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ce59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56d00cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59640/0 (socket says 172.21.15.52:59640) 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ce59c640 1 -- 172.21.15.52:0/2084202966 learned_addr learned my addr 172.21.15.52:0/2084202966 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56cdd9b640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d00b30b0 0x7f56d01510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ced9d640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d00b3530 0x7f56d0151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:54.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ce59c640 1 -- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d00b3530 msgr2=0x7f56d0151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ce59c640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d00b3530 0x7f56d0151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ce59c640 1 -- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d00b30b0 msgr2=0x7f56d01510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ce59c640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d00b30b0 0x7f56d01510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ce59c640 1 -- 172.21.15.52:0/2084202966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d00c8fa0 con 0x7f56b8003a30 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56cdd9b640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d00b30b0 0x7f56d01510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ced9d640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d00b3530 0x7f56d0151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.147+0000 7f56ce59c640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56d00cd390 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f56c0001870 tx=0x7f56c0001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.148+0000 7f56beffd640 1 -- 172.21.15.52:0/2084202966 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c003f070 con 0x7f56b8003a30 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.148+0000 7f56beffd640 1 -- 172.21.15.52:0/2084202966 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c0003c70 con 0x7f56b8003a30 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.148+0000 7f56beffd640 1 -- 172.21.15.52:0/2084202966 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0037040 con 0x7f56b8003a30 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.148+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d00c9280 con 0x7f56b8003a30 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.148+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d00c9840 con 0x7f56b8003a30 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.149+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d00748d0 con 0x7f56b8003a30 2024-05-11T01:13:54.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.149+0000 7f56beffd640 1 -- 172.21.15.52:0/2084202966 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f56c0007220 con 0x7f56b8003a30 2024-05-11T01:13:54.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.150+0000 7f56beffd640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56b00762d0 0x7f56b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:54.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.150+0000 7f56beffd640 1 -- 172.21.15.52:0/2084202966 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6144+0+0 (secure 0 0 0) 0x7f56c00b7510 con 0x7f56b8003a30 2024-05-11T01:13:54.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.150+0000 7f56cdd9b640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56b00762d0 0x7f56b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:54.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.151+0000 7f56cdd9b640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56b00762d0 0x7f56b0078790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f56d006efa0 tx=0x7f56c4008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:54.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.154+0000 7f56beffd640 1 -- 172.21.15.52:0/2084202966 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f56d00748d0 con 0x7f56b8003a30 2024-05-11T01:13:54.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:54.357+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f56d0069780 con 0x7f56b8003a30 2024-05-11T01:13:55.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.406+0000 7f56beffd640 1 -- 172.21.15.52:0/2084202966 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) ==== 97+0+0 (secure 0 0 0) 0x7f56c0080cc0 con 0x7f56b8003a30 2024-05-11T01:13:55.406 INFO:teuthology.orchestra.run.smithi052.stderr: removed all blocklist entries 2024-05-11T01:13:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56b00762d0 msgr2=0x7f56b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56b00762d0 0x7f56b0078790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f56d006efa0 tx=0x7f56c4008040 comp rx=0 tx=0).stop 2024-05-11T01:13:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 msgr2=0x7f56d00cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56d00cd390 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f56c0001870 tx=0x7f56c0001110 comp rx=0 tx=0).stop 2024-05-11T01:13:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 shutdown_connections 2024-05-11T01:13:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56b00762d0 0x7f56b0078790 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d00b3530 0x7f56d0151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d00b30b0 0x7f56d01510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 --2- 172.21.15.52:0/2084202966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56b8003a30 0x7f56d00cd390 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 >> 172.21.15.52:0/2084202966 conn(0x7f56d008eb70 msgr2=0x7f56d00b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.409+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 shutdown_connections 2024-05-11T01:13:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.410+0000 7f56d4a12640 1 -- 172.21.15.52:0/2084202966 wait complete. 2024-05-11T01:13:55.423 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-11T01:13:55.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.533+0000 7f0c36ca0640 1 Processor -- start 2024-05-11T01:13:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.533+0000 7f0c36ca0640 1 -- start start 2024-05-11T01:13:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.534+0000 7f0c36ca0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0c3015d3b0 0x7f0c300b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.534+0000 7f0c36ca0640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c30072a70 con 0x7f0c3015c8c0 2024-05-11T01:13:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.534+0000 7f0c36ca0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c30072d50 con 0x7f0c300722d0 2024-05-11T01:13:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.534+0000 7f0c36ca0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f0c30073030 con 0x7f0c3015d3b0 2024-05-11T01:13:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.534+0000 7f0c34a3c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f0c3015c8c0 0x7f0c30073af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:47968/0 (socket says 172.21.15.52:47968) 2024-05-11T01:13:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.534+0000 7f0c34a3c640 1 -- 172.21.15.52:0/498892550 learned_addr learned my addr 172.21.15.52:0/498892550 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.534+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1983234281 0 0) 0x7f0c30072a70 con 0x7f0c3015c8c0 2024-05-11T01:13:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c1c003520 con 0x7f0c3015c8c0 2024-05-11T01:13:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c3523d640 1 --2- 172.21.15.52:0/498892550 >> v2:172.21.15.119:3300/0 conn(0x7f0c3015d3b0 0x7f0c300b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1245234054 0 0) 0x7f0c1c003520 con 0x7f0c3015c8c0 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 >> v2:172.21.15.119:3300/0 conn(0x7f0c3015d3b0 msgr2=0x7f0c300b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 --2- 172.21.15.52:0/498892550 >> v2:172.21.15.119:3300/0 conn(0x7f0c3015d3b0 0x7f0c300b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 >> v1:172.21.15.119:6789/0 conn(0x7f0c300722d0 legacy=0x7f0c3015cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c30073310 con 0x7f0c3015c8c0 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f0c200016f0 con 0x7f0c3015c8c0 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 >> v1:172.21.15.52:6789/0 conn(0x7f0c3015c8c0 legacy=0x7f0c30073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c1c004800 0x7f0c1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c1c0052e0 0x7f0c1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30073030 con 0x7f0c1c003a50 2024-05-11T01:13:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c00c1b0 con 0x7f0c1c0052e0 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1c00c3b0 con 0x7f0c1c004800 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c34a3c640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c3523d640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c1c0052e0 0x7f0c1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2ffff640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c1c004800 0x7f0c1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c34a3c640 1 -- 172.21.15.52:0/498892550 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c1c004800 msgr2=0x7f0c1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c34a3c640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c1c004800 0x7f0c1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c34a3c640 1 -- 172.21.15.52:0/498892550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c1c0052e0 msgr2=0x7f0c1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c34a3c640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c1c0052e0 0x7f0c1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c34a3c640 1 -- 172.21.15.52:0/498892550 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c003520 con 0x7f0c1c003a50 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c3523d640 1 -- 172.21.15.52:0/498892550 reap_dead start 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.535+0000 7f0c2ffff640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c1c004800 0x7f0c1c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.536+0000 7f0c34a3c640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c1c003e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0c20001050 tx=0x7f0c1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b4d7e39cc224e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.536+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20007d10 con 0x7f0c1c003a50 2024-05-11T01:13:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.536+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c20005ad0 con 0x7f0c1c003a50 2024-05-11T01:13:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.536+0000 7f0c2f7fe640 1 -- 172.21.15.52:0/498892550 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20007070 con 0x7f0c1c003a50 2024-05-11T01:13:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.537+0000 7f0c36ca0640 1 -- 172.21.15.52:0/498892550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 msgr2=0x7f0c1c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.537+0000 7f0c36ca0640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c1c003e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0c20001050 tx=0x7f0c1c005830 comp rx=0 tx=0).stop 2024-05-11T01:13:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.537+0000 7f0c36ca0640 1 -- 172.21.15.52:0/498892550 shutdown_connections 2024-05-11T01:13:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.537+0000 7f0c36ca0640 1 --2- 172.21.15.52:0/498892550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c1c003e70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.537+0000 7f0c36ca0640 1 -- 172.21.15.52:0/498892550 >> 172.21.15.52:0/498892550 conn(0x7f0c30075c00 msgr2=0x7f0c3006c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.537+0000 7f0c36ca0640 1 -- 172.21.15.52:0/498892550 shutdown_connections 2024-05-11T01:13:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.537+0000 7f0c36ca0640 1 -- 172.21.15.52:0/498892550 wait complete. 2024-05-11T01:13:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.538+0000 7f0c36ca0640 1 Processor -- start 2024-05-11T01:13:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.538+0000 7f0c36ca0640 1 -- start start 2024-05-11T01:13:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c36ca0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c3014f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c36ca0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c301f3810 0x7f0c3014e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c36ca0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c3015b730 0x7f0c301efe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c36ca0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3015cca0 con 0x7f0c1c003a50 2024-05-11T01:13:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c36ca0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30072760 con 0x7f0c301f3810 2024-05-11T01:13:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c36ca0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3015ce20 con 0x7f0c3015b730 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c34a3c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c3014f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c34a3c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c3014f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59650/0 (socket says 172.21.15.52:59650) 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c34a3c640 1 -- 172.21.15.52:0/1014130011 learned_addr learned my addr 172.21.15.52:0/1014130011 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c3523d640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c3015b730 0x7f0c301efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c2ffff640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c301f3810 0x7f0c3014e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c34a3c640 1 -- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c3015b730 msgr2=0x7f0c301efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c34a3c640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c3015b730 0x7f0c301efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c34a3c640 1 -- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c301f3810 msgr2=0x7f0c3014e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c34a3c640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c301f3810 0x7f0c3014e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c34a3c640 1 -- 172.21.15.52:0/1014130011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c301f3050 con 0x7f0c1c003a50 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c3523d640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c3015b730 0x7f0c301efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.539+0000 7f0c2ffff640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c301f3810 0x7f0c3014e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.540+0000 7f0c34a3c640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c3014f2b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c300 tx=0x7f0c20000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.540+0000 7f0c2d7fa640 1 -- 172.21.15.52:0/1014130011 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20041070 con 0x7f0c1c003a50 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.540+0000 7f0c2d7fa640 1 -- 172.21.15.52:0/1014130011 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c200047e0 con 0x7f0c1c003a50 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.540+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3014fe50 con 0x7f0c1c003a50 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.540+0000 7f0c2d7fa640 1 -- 172.21.15.52:0/1014130011 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20037040 con 0x7f0c1c003a50 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.540+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3014fb70 con 0x7f0c1c003a50 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.541+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c300696a0 con 0x7f0c1c003a50 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.542+0000 7f0c2d7fa640 1 -- 172.21.15.52:0/1014130011 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f0c20004bd0 con 0x7f0c1c003a50 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.542+0000 7f0c2d7fa640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0c10076300 0x7f0c100787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.542+0000 7f0c2d7fa640 1 -- 172.21.15.52:0/1014130011 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5413+0+0 (secure 0 0 0) 0x7f0c200b6680 con 0x7f0c1c003a50 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.542+0000 7f0c2ffff640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0c10076300 0x7f0c100787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.543+0000 7f0c2ffff640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0c10076300 0x7f0c100787c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c18007fd0 tx=0x7f0c18009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:55.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.547+0000 7f0c2d7fa640 1 -- 172.21.15.52:0/1014130011 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0c20080100 con 0x7f0c1c003a50 2024-05-11T01:13:55.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.814+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0c300cd4c0 con 0x7f0c1c003a50 2024-05-11T01:13:55.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.814+0000 7f0c2d7fa640 1 -- 172.21.15.52:0/1014130011 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f0c2007fa00 con 0x7f0c1c003a50 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: "auth_dump": [ 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.a", 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mP3rNEhAAP5nXdayem1rp+TLkIpVbXg==", 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.814 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.b", 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5m7H/HHhAAFque3IN7APQjExckFVQiQg==", 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.c", 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mmQINGhAAyEyoarOUTjK64GfK5PZ5bQ==", 2024-05-11T01:13:55.815 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mds.d", 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mXGYNJhAA9loGi/oMhZT+LWktBsVhfw==", 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:13:55.816 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.0", 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAfwz5mJZqrDxAAq3xCv8Pf2DPOEL2Tovrbeg==", 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.1", 2024-05-11T01:13:55.817 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAiwz5mkbxtDhAAHD/TxmE4UfGOWvArcQsdNA==", 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.2", 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAlwz5mgOegDBAAKU+gnyyeIwLuvXRfzOD/ZQ==", 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.3", 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAowz5mERnWChAAZVaafqVtHNgrii/OZSYXdg==", 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.4", 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAuwz5mbL3bERAAO7anuNWH7NtNl5//BukL8g==", 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.819 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.5", 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAxwz5myrFsEBAAvGSIqmyuovYeRCSoDNTLxg==", 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.820 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.6", 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA0wz5mythuDxAAJh/fQNHjv0kVV420jaSiyQ==", 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "osd.7", 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQA3wz5mSAXGDRAAoWVOOP2dLYGaGsue3vxWwA==", 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow profile osd", 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile osd", 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.821 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.0", 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mUO0yKhAAfS2pcT6nSB+NrJccryMO8A==", 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow r", 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.1", 2024-05-11T01:13:55.822 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5m3AodMxAAXArXFLqTR6kBG8KTzop93w==", 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.2", 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5maFjoLxAAwD/wgFjjG9lbeRLUtV4BRQ==", 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-11T01:13:55.823 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.3", 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mEb+wOBAAI8i4rWMD8GWMUf6p0HICRA==", 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow", 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow r", 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow rw", 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow rwx" 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.824 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "client.admin", 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAcwz5mmFmRMhAAIcbYdP/83D06BB/z3mJCTw==", 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr": "allow *", 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow *", 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mgr.x", 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5mj0l4DxAAnDj4+dETpUu/ykNczo5dvA==", 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile mgr", 2024-05-11T01:13:55.825 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: "entity": "mgr.y", 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: "key": "AQAdwz5medHbCxAASin8Gv729ZztriQmuTTTYQ==", 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": { 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": "allow *", 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: "mon": "allow profile mgr", 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: "osd": "allow *" 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-05-11T01:13:55.826 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:13:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0c10076300 msgr2=0x7f0c100787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0c10076300 0x7f0c100787c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c18007fd0 tx=0x7f0c18009040 comp rx=0 tx=0).stop 2024-05-11T01:13:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 msgr2=0x7f0c3014f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c3014f2b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c300 tx=0x7f0c20000fe0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 shutdown_connections 2024-05-11T01:13:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f0c10076300 0x7f0c100787c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c3015b730 0x7f0c301efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c301f3810 0x7f0c3014e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 --2- 172.21.15.52:0/1014130011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c1c003a50 0x7f0c3014f2b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 >> 172.21.15.52:0/1014130011 conn(0x7f0c30075c00 msgr2=0x7f0c300b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 shutdown_connections 2024-05-11T01:13:55.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.816+0000 7f0c36ca0640 1 -- 172.21.15.52:0/1014130011 wait complete. 2024-05-11T01:13:55.829 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T01:13:55.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.934+0000 7f41464b4640 1 Processor -- start 2024-05-11T01:13:55.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.934+0000 7f41464b4640 1 -- start start 2024-05-11T01:13:55.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f41464b4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f414015f420 0x7f414006d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f41464b4640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f41400b2400 0x7f4140070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f41464b4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41400d1240 con 0x7f4140071470 2024-05-11T01:13:55.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f41464b4640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f414015f800 con 0x7f41400b2400 2024-05-11T01:13:55.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f41464b4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f41400d14b0 con 0x7f414015f420 2024-05-11T01:13:55.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f4137fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f41400b2400 0x7f4140070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f4137fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f41400b2400 0x7f4140070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59658/0 (socket says 172.21.15.52:59658) 2024-05-11T01:13:55.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f4137fff640 1 -- 172.21.15.52:0/3536263139 learned_addr learned my addr 172.21.15.52:0/3536263139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:55.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f4137fff640 1 -- 172.21.15.52:0/3536263139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f414015f420 msgr2=0x7f414006d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:55.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f413ffff640 1 --2- 172.21.15.52:0/3536263139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f414015f420 0x7f414006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f4137fff640 1 --2- 172.21.15.52:0/3536263139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f414015f420 0x7f414006d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f4137fff640 1 -- 172.21.15.52:0/3536263139 >> v1:172.21.15.119:6789/0 conn(0x7f4140071470 legacy=0x7f41400cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.935+0000 7f4137fff640 1 -- 172.21.15.52:0/3536263139 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41400d1790 con 0x7f41400b2400 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.936+0000 7f4137fff640 1 --2- 172.21.15.52:0/3536263139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41400b2400 0x7f4140070f30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4128009870 tx=0x7f412802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a81a1e03f4f6bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.936+0000 7f413f7fe640 1 -- 172.21.15.52:0/3536263139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128037070 con 0x7f41400b2400 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.936+0000 7f413f7fe640 1 -- 172.21.15.52:0/3536263139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4128032440 con 0x7f41400b2400 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.936+0000 7f413f7fe640 1 -- 172.21.15.52:0/3536263139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128032810 con 0x7f41400b2400 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.937+0000 7f41464b4640 1 -- 172.21.15.52:0/3536263139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41400b2400 msgr2=0x7f4140070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.937+0000 7f41464b4640 1 --2- 172.21.15.52:0/3536263139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41400b2400 0x7f4140070f30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4128009870 tx=0x7f412802a1c0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.937+0000 7f41464b4640 1 -- 172.21.15.52:0/3536263139 shutdown_connections 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.937+0000 7f41464b4640 1 --2- 172.21.15.52:0/3536263139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41400b2400 0x7f4140070f30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.937+0000 7f41464b4640 1 --2- 172.21.15.52:0/3536263139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f414015f420 0x7f414006d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.937+0000 7f41464b4640 1 -- 172.21.15.52:0/3536263139 >> 172.21.15.52:0/3536263139 conn(0x7f41400d3080 msgr2=0x7f41400d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:55.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.937+0000 7f41464b4640 1 -- 172.21.15.52:0/3536263139 shutdown_connections 2024-05-11T01:13:55.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.937+0000 7f41464b4640 1 -- 172.21.15.52:0/3536263139 wait complete. 2024-05-11T01:13:55.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.938+0000 7f41464b4640 1 Processor -- start 2024-05-11T01:13:55.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.938+0000 7f41464b4640 1 -- start start 2024-05-11T01:13:55.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f41464b4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4140071470 0x7f41401550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f41464b4640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41400b2400 0x7f4140157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f41464b4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f414015f420 0x7f41401612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f41464b4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f414006ddc0 con 0x7f414015f420 2024-05-11T01:13:55.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f41464b4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f41400d1d50 con 0x7f4140071470 2024-05-11T01:13:55.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f41464b4640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f414006df40 con 0x7f41400b2400 2024-05-11T01:13:55.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4144a51640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f414015f420 0x7f41401612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4144a51640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f414015f420 0x7f41401612d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59664/0 (socket says 172.21.15.52:59664) 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4144a51640 1 -- 172.21.15.52:0/1565975256 learned_addr learned my addr 172.21.15.52:0/1565975256 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f413ffff640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4140071470 0x7f41401550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4137fff640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41400b2400 0x7f4140157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4144a51640 1 -- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41400b2400 msgr2=0x7f4140157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4144a51640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41400b2400 0x7f4140157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4144a51640 1 -- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4140071470 msgr2=0x7f41401550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4144a51640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4140071470 0x7f41401550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4144a51640 1 -- 172.21.15.52:0/1565975256 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4140157c80 con 0x7f414015f420 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f413ffff640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4140071470 0x7f41401550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.939+0000 7f4144a51640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f414015f420 0x7f41401612d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f413800c780 tx=0x7f413800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.940+0000 7f413d7fa640 1 -- 172.21.15.52:0/1565975256 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138004b60 con 0x7f414015f420 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.940+0000 7f413d7fa640 1 -- 172.21.15.52:0/1565975256 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4138004d00 con 0x7f414015f420 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.940+0000 7f413d7fa640 1 -- 172.21.15.52:0/1565975256 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413800aa90 con 0x7f414015f420 2024-05-11T01:13:55.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.940+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4140157f60 con 0x7f414015f420 2024-05-11T01:13:55.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.940+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4140158520 con 0x7f414015f420 2024-05-11T01:13:55.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.941+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41400696a0 con 0x7f414015f420 2024-05-11T01:13:55.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.941+0000 7f413d7fa640 1 -- 172.21.15.52:0/1565975256 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f413800ac30 con 0x7f414015f420 2024-05-11T01:13:55.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.942+0000 7f413d7fa640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f41200762d0 0x7f4120078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:55.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.942+0000 7f413d7fa640 1 -- 172.21.15.52:0/1565975256 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5413+0+0 (secure 0 0 0) 0x7f4138097740 con 0x7f414015f420 2024-05-11T01:13:55.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.942+0000 7f413ffff640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f41200762d0 0x7f4120078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:55.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.943+0000 7f413ffff640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f41200762d0 0x7f4120078790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4130006fd0 tx=0x7f4130008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:55.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:55.946+0000 7f413d7fa640 1 -- 172.21.15.52:0/1565975256 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4138020070 con 0x7f414015f420 2024-05-11T01:13:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.144+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f414006d660 con 0x7f414015f420 2024-05-11T01:13:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.144+0000 7f413d7fa640 1 -- 172.21.15.52:0/1565975256 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) ==== 84+0+65 (secure 0 0 0) 0x7f4138061120 con 0x7f414015f420 2024-05-11T01:13:56.143 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:56.143 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-05-11T01:13:56.143 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-05-11T01:13:56.144 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-05-11T01:13:56.144 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-05-11T01:13:56.144 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:13:56.144 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-05-11T01:13:56.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f41200762d0 msgr2=0x7f4120078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f41200762d0 0x7f4120078790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4130006fd0 tx=0x7f4130008040 comp rx=0 tx=0).stop 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f414015f420 msgr2=0x7f41401612d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f414015f420 0x7f41401612d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f413800c780 tx=0x7f413800cc40 comp rx=0 tx=0).stop 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 shutdown_connections 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f41200762d0 0x7f4120078790 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f414015f420 0x7f41401612d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41400b2400 0x7f4140157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 --2- 172.21.15.52:0/1565975256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4140071470 0x7f41401550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 >> 172.21.15.52:0/1565975256 conn(0x7f41400d3080 msgr2=0x7f414006e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 shutdown_connections 2024-05-11T01:13:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.146+0000 7f41464b4640 1 -- 172.21.15.52:0/1565975256 wait complete. 2024-05-11T01:13:56.157 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T01:13:56.157 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T01:13:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.265+0000 7fc718845640 1 Processor -- start 2024-05-11T01:13:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.265+0000 7fc718845640 1 -- start start 2024-05-11T01:13:56.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.266+0000 7fc718845640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc71415d0b0 0x7fc714072120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:56.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.266+0000 7fc718845640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 0x7fc71406e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.266+0000 7fc718845640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc714160d50 con 0x7fc714073f00 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.266+0000 7fc718845640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fc714161030 con 0x7fc71415d0b0 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.266+0000 7fc718845640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc714161310 con 0x7fc7140cba90 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.266+0000 7fc71259c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fc71415d0b0 0x7fc714072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.266+0000 7fc711d9b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc714073f00 0x7fc7140cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:47980/0 (socket says 172.21.15.52:47980) 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.266+0000 7fc711d9b640 1 -- 172.21.15.52:0/2128063884 learned_addr learned my addr 172.21.15.52:0/2128063884 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.267+0000 7fc712d9d640 1 --2- 172.21.15.52:0/2128063884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 0x7fc71406e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.267+0000 7fc71259c640 1 -- 172.21.15.52:0/2128063884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 msgr2=0x7fc71406e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.267+0000 7fc71259c640 1 --2- 172.21.15.52:0/2128063884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 0x7fc71406e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.267+0000 7fc71259c640 1 -- 172.21.15.52:0/2128063884 >> v1:172.21.15.52:6789/0 conn(0x7fc714073f00 legacy=0x7fc7140cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.267+0000 7fc71159a640 1 -- 172.21.15.52:0/2128063884 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2105890780 0 0) 0x7fc714160d50 con 0x7fc714073f00 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.267+0000 7fc71259c640 1 -- 172.21.15.52:0/2128063884 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7141615f0 con 0x7fc71415d0b0 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.267+0000 7fc71259c640 1 --2- 172.21.15.52:0/2128063884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc71415d0b0 0x7fc714072120 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc7040097a0 tx=0x7fc70402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0115d07a7dd673 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc71159a640 1 -- 172.21.15.52:0/2128063884 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704037070 con 0x7fc71415d0b0 2024-05-11T01:13:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc71159a640 1 -- 172.21.15.52:0/2128063884 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc704032440 con 0x7fc71415d0b0 2024-05-11T01:13:56.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc71159a640 1 -- 172.21.15.52:0/2128063884 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704032810 con 0x7fc71415d0b0 2024-05-11T01:13:56.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc718845640 1 -- 172.21.15.52:0/2128063884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc71415d0b0 msgr2=0x7fc714072120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:56.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc718845640 1 --2- 172.21.15.52:0/2128063884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc71415d0b0 0x7fc714072120 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc7040097a0 tx=0x7fc70402a180 comp rx=0 tx=0).stop 2024-05-11T01:13:56.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc718845640 1 -- 172.21.15.52:0/2128063884 shutdown_connections 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc718845640 1 --2- 172.21.15.52:0/2128063884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 0x7fc71406e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc718845640 1 --2- 172.21.15.52:0/2128063884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc71415d0b0 0x7fc714072120 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc718845640 1 -- 172.21.15.52:0/2128063884 >> 172.21.15.52:0/2128063884 conn(0x7fc7140cd7b0 msgr2=0x7fc7140cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc718845640 1 -- 172.21.15.52:0/2128063884 shutdown_connections 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.268+0000 7fc718845640 1 -- 172.21.15.52:0/2128063884 wait complete. 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.269+0000 7fc718845640 1 Processor -- start 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc718845640 1 -- start start 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc718845640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714073f00 0x7fc714153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc718845640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 0x7fc7140c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc718845640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc71415d0b0 0x7fc714147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc718845640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc714072660 con 0x7fc714073f00 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc718845640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc71415e4a0 con 0x7fc71415d0b0 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc718845640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc7140727e0 con 0x7fc7140cba90 2024-05-11T01:13:56.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc71259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714073f00 0x7fc714153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc71259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714073f00 0x7fc714153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59694/0 (socket says 172.21.15.52:59694) 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.270+0000 7fc71259c640 1 -- 172.21.15.52:0/2242110574 learned_addr learned my addr 172.21.15.52:0/2242110574 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc711d9b640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 0x7fc7140c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc712d9d640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc71415d0b0 0x7fc714147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc71259c640 1 -- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 msgr2=0x7fc7140c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc71259c640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 0x7fc7140c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc71259c640 1 -- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc71415d0b0 msgr2=0x7fc714147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc71259c640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc71415d0b0 0x7fc714147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc71259c640 1 -- 172.21.15.52:0/2242110574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc714147570 con 0x7fc714073f00 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc711d9b640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 0x7fc7140c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc71259c640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714073f00 0x7fc714153100 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc704000c00 tx=0x7fc7040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc702ffd640 1 -- 172.21.15.52:0/2242110574 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704037040 con 0x7fc714073f00 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc702ffd640 1 -- 172.21.15.52:0/2242110574 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc704042070 con 0x7fc714073f00 2024-05-11T01:13:56.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7140c8550 con 0x7fc714073f00 2024-05-11T01:13:56.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc702ffd640 1 -- 172.21.15.52:0/2242110574 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704030bb0 con 0x7fc714073f00 2024-05-11T01:13:56.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.271+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7140c8b10 con 0x7fc714073f00 2024-05-11T01:13:56.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.272+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc714069780 con 0x7fc714073f00 2024-05-11T01:13:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.273+0000 7fc702ffd640 1 -- 172.21.15.52:0/2242110574 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fc704047460 con 0x7fc714073f00 2024-05-11T01:13:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.273+0000 7fc702ffd640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6f40762d0 0x7fc6f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.273+0000 7fc702ffd640 1 -- 172.21.15.52:0/2242110574 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5413+0+0 (secure 0 0 0) 0x7fc7040b8010 con 0x7fc714073f00 2024-05-11T01:13:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.273+0000 7fc711d9b640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6f40762d0 0x7fc6f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.274+0000 7fc711d9b640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6f40762d0 0x7fc6f4078790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc708000f60 tx=0x7fc708007bd0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.277+0000 7fc702ffd640 1 -- 172.21.15.52:0/2242110574 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc704032e20 con 0x7fc714073f00 2024-05-11T01:13:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:56.473+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc7140b1210 con 0x7fc714073f00 2024-05-11T01:13:57.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.409+0000 7fc702ffd640 1 -- 172.21.15.52:0/2242110574 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) ==== 138+0+0 (secure 0 0 0) 0x7fc704081b20 con 0x7fc714073f00 2024-05-11T01:13:57.409 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-05-11T01:13:57.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6f40762d0 msgr2=0x7fc6f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:57.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6f40762d0 0x7fc6f4078790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc708000f60 tx=0x7fc708007bd0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714073f00 msgr2=0x7fc714153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:57.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714073f00 0x7fc714153100 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc704000c00 tx=0x7fc7040046c0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 shutdown_connections 2024-05-11T01:13:57.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6f40762d0 0x7fc6f4078790 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc71415d0b0 0x7fc714147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc7140cba90 0x7fc7140c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 --2- 172.21.15.52:0/2242110574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc714073f00 0x7fc714153100 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.414+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 >> 172.21.15.52:0/2242110574 conn(0x7fc7140cd7b0 msgr2=0x7fc71415ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:57.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.416+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 shutdown_connections 2024-05-11T01:13:57.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.416+0000 7fc718845640 1 -- 172.21.15.52:0/2242110574 wait complete. 2024-05-11T01:13:57.431 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T01:13:57.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.548+0000 7faa807da640 1 Processor -- start 2024-05-11T01:13:57.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.548+0000 7faa807da640 1 -- start start 2024-05-11T01:13:57.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.549+0000 7faa807da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa7815f6f0 0x7faa7806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:57.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.549+0000 7faa807da640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa780cdd60 0x7faa781630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:57.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.549+0000 7faa807da640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa780d12d0 con 0x7faa780d2330 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.549+0000 7faa807da640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7faa7806dc10 con 0x7faa780cdd60 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.549+0000 7faa807da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7faa780d1570 con 0x7faa7815f6f0 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7dd75640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7faa780d2330 0x7faa780cd650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:47988/0 (socket says 172.21.15.52:47988) 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7dd75640 1 -- 172.21.15.52:0/2965640531 learned_addr learned my addr 172.21.15.52:0/2965640531 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7ed77640 1 --2- 172.21.15.52:0/2965640531 >> v2:172.21.15.119:3300/0 conn(0x7faa780cdd60 0x7faa781630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7e576640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa7815f6f0 0x7faa7806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72401577 0 0) 0x7faa780d12d0 con 0x7faa780d2330 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa64003520 con 0x7faa780d2330 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3264591175 0 0) 0x7faa64003520 con 0x7faa780d2330 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa7815f6f0 msgr2=0x7faa7806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa7815f6f0 0x7faa7806d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 >> v2:172.21.15.119:3300/0 conn(0x7faa780cdd60 msgr2=0x7faa781630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:57.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 --2- 172.21.15.52:0/2965640531 >> v2:172.21.15.119:3300/0 conn(0x7faa780cdd60 0x7faa781630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa780d1850 con 0x7faa780d2330 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7faa680016f0 con 0x7faa780d2330 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 >> v1:172.21.15.52:6789/0 conn(0x7faa780d2330 legacy=0x7faa780cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa64003a00 0x7faa64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa64004800 0x7faa64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa640051c0 0x7faa6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faa7806dc10 con 0x7faa64003a00 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa780d1570 con 0x7faa64004800 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7faa6400bf00 con 0x7faa640051c0 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7e576640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa64003a00 0x7faa64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.550+0000 7faa7dd75640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa64004800 0x7faa64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.551+0000 7faa7ed77640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa640051c0 0x7faa6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:57.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.551+0000 7faa7e576640 1 -- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa640051c0 msgr2=0x7faa6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.551+0000 7faa7e576640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa640051c0 0x7faa6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.551+0000 7faa7e576640 1 -- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa64004800 msgr2=0x7faa64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.551+0000 7faa7e576640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa64004800 0x7faa64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.551+0000 7faa7e576640 1 -- 172.21.15.52:0/2965640531 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa64003520 con 0x7faa64003a00 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.551+0000 7faa7e576640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa64003a00 0x7faa64003e20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faa74009ac0 tx=0x7faa64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723d0af44242059a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74036070 con 0x7faa64003a00 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa74031400 con 0x7faa64003a00 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa7d574640 1 -- 172.21.15.52:0/2965640531 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74009690 con 0x7faa64003a00 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa807da640 1 -- 172.21.15.52:0/2965640531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa64003a00 msgr2=0x7faa64003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa807da640 1 --2- 172.21.15.52:0/2965640531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa64003a00 0x7faa64003e20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faa74009ac0 tx=0x7faa64005830 comp rx=0 tx=0).stop 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa7ed77640 1 -- 172.21.15.52:0/2965640531 reap_dead start 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa807da640 1 -- 172.21.15.52:0/2965640531 shutdown_connections 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa807da640 1 -- 172.21.15.52:0/2965640531 >> 172.21.15.52:0/2965640531 conn(0x7faa780d3000 msgr2=0x7faa780d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:57.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa807da640 1 -- 172.21.15.52:0/2965640531 shutdown_connections 2024-05-11T01:13:57.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.552+0000 7faa807da640 1 -- 172.21.15.52:0/2965640531 wait complete. 2024-05-11T01:13:57.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.553+0000 7faa807da640 1 Processor -- start 2024-05-11T01:13:57.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa807da640 1 -- start start 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa807da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa64003a00 0x7faa7815c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa807da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa780cc410 0x7faa780cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa807da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa781f5290 0x7faa781f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa807da640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faa780d3b90 con 0x7faa781f5290 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa807da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa780d1e10 con 0x7faa64003a00 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa807da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7faa780ce1f0 con 0x7faa780cc410 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa7ed77640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa781f5290 0x7faa781f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa7ed77640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa781f5290 0x7faa781f5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59716/0 (socket says 172.21.15.52:59716) 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.554+0000 7faa7ed77640 1 -- 172.21.15.52:0/1924278171 learned_addr learned my addr 172.21.15.52:0/1924278171 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa7e576640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa64003a00 0x7faa7815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:57.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa7dd75640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa780cc410 0x7faa780cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa7ed77640 1 -- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa780cc410 msgr2=0x7faa780cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa7ed77640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa780cc410 0x7faa780cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa7ed77640 1 -- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa64003a00 msgr2=0x7faa7815c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa7ed77640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa64003a00 0x7faa7815c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa7ed77640 1 -- 172.21.15.52:0/1924278171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa780cb370 con 0x7faa781f5290 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa7ed77640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa781f5290 0x7faa781f5740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7faa78073730 tx=0x7faa7000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa6effd640 1 -- 172.21.15.52:0/1924278171 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7000fce0 con 0x7faa781f5290 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa6effd640 1 -- 172.21.15.52:0/1924278171 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7000fe80 con 0x7faa781f5290 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa6effd640 1 -- 172.21.15.52:0/1924278171 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70015670 con 0x7faa781f5290 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa780cb650 con 0x7faa781f5290 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.555+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa780cbc10 con 0x7faa781f5290 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.556+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa78069c00 con 0x7faa781f5290 2024-05-11T01:13:57.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.557+0000 7faa6effd640 1 -- 172.21.15.52:0/1924278171 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7faa70015810 con 0x7faa781f5290 2024-05-11T01:13:57.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.557+0000 7faa6effd640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa5c076300 0x7faa5c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:57.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.558+0000 7faa6effd640 1 -- 172.21.15.52:0/1924278171 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5783+0+0 (secure 0 0 0) 0x7faa70098380 con 0x7faa781f5290 2024-05-11T01:13:57.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.558+0000 7faa7e576640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa5c076300 0x7faa5c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:57.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.558+0000 7faa7e576640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa5c076300 0x7faa5c0787c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa74033000 tx=0x7faa74008030 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:57.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.562+0000 7faa6effd640 1 -- 172.21.15.52:0/1924278171 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faa70019030 con 0x7faa781f5290 2024-05-11T01:13:57.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:57.753+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faa7806e950 con 0x7faa781f5290 2024-05-11T01:13:58.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.415+0000 7faa6effd640 1 -- 172.21.15.52:0/1924278171 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) ==== 170+0+0 (secure 0 0 0) 0x7faa70010040 con 0x7faa781f5290 2024-05-11T01:13:58.415 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-05-11T01:13:58.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa5c076300 msgr2=0x7faa5c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa5c076300 0x7faa5c0787c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa74033000 tx=0x7faa74008030 comp rx=0 tx=0).stop 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa781f5290 msgr2=0x7faa781f5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa781f5290 0x7faa781f5740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7faa78073730 tx=0x7faa7000f1b0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 shutdown_connections 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa5c076300 0x7faa5c0787c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa781f5290 0x7faa781f5740 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa780cc410 0x7faa780cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 --2- 172.21.15.52:0/1924278171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa64003a00 0x7faa7815c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 >> 172.21.15.52:0/1924278171 conn(0x7faa780d3000 msgr2=0x7faa780681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 shutdown_connections 2024-05-11T01:13:58.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.419+0000 7faa807da640 1 -- 172.21.15.52:0/1924278171 wait complete. 2024-05-11T01:13:58.444 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T01:13:58.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.555+0000 7f5e1b064640 1 Processor -- start 2024-05-11T01:13:58.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.555+0000 7f5e1b064640 1 -- start start 2024-05-11T01:13:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.555+0000 7f5e1b064640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5e1415f6f0 0x7f5e1406d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.555+0000 7f5e1b064640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e140d2330 0x7f5e140cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e1b064640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e140d12d0 con 0x7f5e140cdb90 2024-05-11T01:13:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e1b064640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5e1406dc10 con 0x7f5e1415f6f0 2024-05-11T01:13:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e1b064640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f5e140d1570 con 0x7f5e140d2330 2024-05-11T01:13:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e18e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5e1415f6f0 0x7f5e1406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e18e00640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5e1415f6f0 0x7f5e1406d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59732/0 (socket says 172.21.15.52:59732) 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e18e00640 1 -- 172.21.15.52:0/3460793939 learned_addr learned my addr 172.21.15.52:0/3460793939 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e13fff640 1 --2- 172.21.15.52:0/3460793939 >> v2:172.21.15.119:3300/0 conn(0x7f5e140d2330 0x7f5e140cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e18e00640 1 -- 172.21.15.52:0/3460793939 >> v2:172.21.15.119:3300/0 conn(0x7f5e140d2330 msgr2=0x7f5e140cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e18e00640 1 --2- 172.21.15.52:0/3460793939 >> v2:172.21.15.119:3300/0 conn(0x7f5e140d2330 0x7f5e140cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e18e00640 1 -- 172.21.15.52:0/3460793939 >> v1:172.21.15.119:6789/0 conn(0x7f5e140cdb90 legacy=0x7f5e140cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.556+0000 7f5e18e00640 1 -- 172.21.15.52:0/3460793939 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e140d1850 con 0x7f5e1415f6f0 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.557+0000 7f5e18e00640 1 --2- 172.21.15.52:0/3460793939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1415f6f0 0x7f5e1406d630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009e50 tx=0x7f5e0c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6313dc0bcd91966 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.557+0000 7f5e137fe640 1 -- 172.21.15.52:0/3460793939 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c036070 con 0x7f5e1415f6f0 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.557+0000 7f5e137fe640 1 -- 172.21.15.52:0/3460793939 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e0c031600 con 0x7f5e1415f6f0 2024-05-11T01:13:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.557+0000 7f5e137fe640 1 -- 172.21.15.52:0/3460793939 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c0319d0 con 0x7f5e1415f6f0 2024-05-11T01:13:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.558+0000 7f5e1b064640 1 -- 172.21.15.52:0/3460793939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1415f6f0 msgr2=0x7f5e1406d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.558+0000 7f5e1b064640 1 --2- 172.21.15.52:0/3460793939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1415f6f0 0x7f5e1406d630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009e50 tx=0x7f5e0c02a760 comp rx=0 tx=0).stop 2024-05-11T01:13:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.558+0000 7f5e1b064640 1 -- 172.21.15.52:0/3460793939 shutdown_connections 2024-05-11T01:13:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.558+0000 7f5e1b064640 1 --2- 172.21.15.52:0/3460793939 >> v2:172.21.15.119:3300/0 conn(0x7f5e140d2330 0x7f5e140cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.558+0000 7f5e1b064640 1 --2- 172.21.15.52:0/3460793939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1415f6f0 0x7f5e1406d630 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.558+0000 7f5e1b064640 1 -- 172.21.15.52:0/3460793939 >> 172.21.15.52:0/3460793939 conn(0x7f5e140d3000 msgr2=0x7f5e140d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.558+0000 7f5e1b064640 1 -- 172.21.15.52:0/3460793939 shutdown_connections 2024-05-11T01:13:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.558+0000 7f5e1b064640 1 -- 172.21.15.52:0/3460793939 wait complete. 2024-05-11T01:13:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.559+0000 7f5e1b064640 1 Processor -- start 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.559+0000 7f5e1b064640 1 -- start start 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e1b064640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e140cdb90 0x7f5e140ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e1b064640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e140d2330 0x7f5e141f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e1b064640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e14157f20 0x7f5e1414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e1b064640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5e140d3b90 con 0x7f5e14157f20 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e1b064640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e140d1e10 con 0x7f5e140d2330 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e1b064640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5e140ce000 con 0x7f5e140cdb90 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e19601640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e14157f20 0x7f5e1414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e19601640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e14157f20 0x7f5e1414b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59740/0 (socket says 172.21.15.52:59740) 2024-05-11T01:13:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e19601640 1 -- 172.21.15.52:0/2280406882 learned_addr learned my addr 172.21.15.52:0/2280406882 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e13fff640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e140d2330 0x7f5e141f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e18e00640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e140cdb90 0x7f5e140ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e19601640 1 -- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e140cdb90 msgr2=0x7f5e140ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e19601640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e140cdb90 0x7f5e140ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e19601640 1 -- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e140d2330 msgr2=0x7f5e141f9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e19601640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e140d2330 0x7f5e141f9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.560+0000 7f5e19601640 1 -- 172.21.15.52:0/2280406882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e141f9db0 con 0x7f5e14157f20 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.561+0000 7f5e19601640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e14157f20 0x7f5e1414b3b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400c400 tx=0x7f5e0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.561+0000 7f5e117fa640 1 -- 172.21.15.52:0/2280406882 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04017070 con 0x7f5e14157f20 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.561+0000 7f5e117fa640 1 -- 172.21.15.52:0/2280406882 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e040052d0 con 0x7f5e14157f20 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.561+0000 7f5e117fa640 1 -- 172.21.15.52:0/2280406882 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e040037f0 con 0x7f5e14157f20 2024-05-11T01:13:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.561+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e14158400 con 0x7f5e14157f20 2024-05-11T01:13:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.561+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e141589c0 con 0x7f5e14157f20 2024-05-11T01:13:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.562+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e14069c00 con 0x7f5e14157f20 2024-05-11T01:13:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.563+0000 7f5e117fa640 1 -- 172.21.15.52:0/2280406882 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f5e0401a070 con 0x7f5e14157f20 2024-05-11T01:13:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.563+0000 7f5e117fa640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5e000762d0 0x7f5e00078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.563+0000 7f5e117fa640 1 -- 172.21.15.52:0/2280406882 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6165+0+0 (secure 0 0 0) 0x7f5e04096a90 con 0x7f5e14157f20 2024-05-11T01:13:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.563+0000 7f5e18e00640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5e000762d0 0x7f5e00078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.564+0000 7f5e18e00640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5e000762d0 0x7f5e00078790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c033000 tx=0x7f5e0c009000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:58.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.568+0000 7f5e117fa640 1 -- 172.21.15.52:0/2280406882 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5e0405fbd0 con 0x7f5e14157f20 2024-05-11T01:13:58.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:58.798+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5e140ce3c0 con 0x7f5e14157f20 2024-05-11T01:13:59.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.417+0000 7f5e117fa640 1 -- 172.21.15.52:0/2280406882 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) ==== 172+0+0 (secure 0 0 0) 0x7f5e04068210 con 0x7f5e14157f20 2024-05-11T01:13:59.416 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 6 and data pool 7 2024-05-11T01:13:59.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5e000762d0 msgr2=0x7f5e00078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5e000762d0 0x7f5e00078790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c033000 tx=0x7f5e0c009000 comp rx=0 tx=0).stop 2024-05-11T01:13:59.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e14157f20 msgr2=0x7f5e1414b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e14157f20 0x7f5e1414b3b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400c400 tx=0x7f5e0400c8c0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 shutdown_connections 2024-05-11T01:13:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5e000762d0 0x7f5e00078790 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e14157f20 0x7f5e1414b3b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e140d2330 0x7f5e141f9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 --2- 172.21.15.52:0/2280406882 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e140cdb90 0x7f5e140ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.421+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 >> 172.21.15.52:0/2280406882 conn(0x7f5e140d3000 msgr2=0x7f5e140b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:59.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.426+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 shutdown_connections 2024-05-11T01:13:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.433+0000 7f5e1b064640 1 -- 172.21.15.52:0/2280406882 wait complete. 2024-05-11T01:13:59.457 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:13:59.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.564+0000 7fe4cbf46640 1 Processor -- start 2024-05-11T01:13:59.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.564+0000 7fe4cbf46640 1 -- start start 2024-05-11T01:13:59.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.565+0000 7fe4cbf46640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe4c40d3330 0x7fe4c40b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.565+0000 7fe4cbf46640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c40d1f90 con 0x7fe4c40d2ea0 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.565+0000 7fe4cbf46640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c40d2270 con 0x7fe4c415f2d0 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.565+0000 7fe4cbf46640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c40d2550 con 0x7fe4c40d3330 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.565+0000 7fe4ca4e3640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fe4c40d3330 0x7fe4c40b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.565+0000 7fe4c94e1640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fe4c40d2ea0 0x7fe4c40b2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:47998/0 (socket says 172.21.15.52:47998) 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.565+0000 7fe4c94e1640 1 -- 172.21.15.52:0/3484772291 learned_addr learned my addr 172.21.15.52:0/3484772291 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.566+0000 7fe4ca4e3640 1 -- 172.21.15.52:0/3484772291 >> v1:172.21.15.119:6789/0 conn(0x7fe4c415f2d0 legacy=0x7fe4c406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.566+0000 7fe4ca4e3640 1 -- 172.21.15.52:0/3484772291 >> v1:172.21.15.52:6789/0 conn(0x7fe4c40d2ea0 legacy=0x7fe4c40b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.566+0000 7fe4ca4e3640 1 -- 172.21.15.52:0/3484772291 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c40d2830 con 0x7fe4c40d3330 2024-05-11T01:13:59.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.566+0000 7fe4c8ce0640 1 -- 172.21.15.52:0/3484772291 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1199326542 0 0) 0x7fe4c40d1f90 con 0x7fe4c40d2ea0 2024-05-11T01:13:59.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.566+0000 7fe4ca4e3640 1 --2- 172.21.15.52:0/3484772291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c40d3330 0x7fe4c40b2d10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009d90 tx=0x7fe4bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71de5cf1b3d82f5a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.566+0000 7fe4c8ce0640 1 -- 172.21.15.52:0/3484772291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc007740 con 0x7fe4c40d3330 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.566+0000 7fe4c8ce0640 1 -- 172.21.15.52:0/3484772291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4bc0078e0 con 0x7fe4c40d3330 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.566+0000 7fe4c8ce0640 1 -- 172.21.15.52:0/3484772291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc007cb0 con 0x7fe4c40d3330 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.567+0000 7fe4cbf46640 1 -- 172.21.15.52:0/3484772291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c40d3330 msgr2=0x7fe4c40b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.567+0000 7fe4cbf46640 1 --2- 172.21.15.52:0/3484772291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c40d3330 0x7fe4c40b2d10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009d90 tx=0x7fe4bc02a660 comp rx=0 tx=0).stop 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.567+0000 7fe4cbf46640 1 -- 172.21.15.52:0/3484772291 shutdown_connections 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.567+0000 7fe4cbf46640 1 --2- 172.21.15.52:0/3484772291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c40d3330 0x7fe4c40b2d10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.567+0000 7fe4cbf46640 1 -- 172.21.15.52:0/3484772291 >> 172.21.15.52:0/3484772291 conn(0x7fe4c409ba20 msgr2=0x7fe4c409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.567+0000 7fe4cbf46640 1 -- 172.21.15.52:0/3484772291 shutdown_connections 2024-05-11T01:13:59.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.567+0000 7fe4cbf46640 1 -- 172.21.15.52:0/3484772291 wait complete. 2024-05-11T01:13:59.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.568+0000 7fe4cbf46640 1 Processor -- start 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.568+0000 7fe4cbf46640 1 -- start start 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4cbf46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c40d2ea0 0x7fe4c4161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4cbf46640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c40d3330 0x7fe4c41f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4cbf46640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c415f2d0 0x7fe4c4160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4cbf46640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c415f6b0 con 0x7fe4c415f2d0 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4cbf46640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c40d3740 con 0x7fe4c40d2ea0 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4cbf46640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c415f830 con 0x7fe4c40d3330 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4ca4e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c415f2d0 0x7fe4c4160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4ca4e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c415f2d0 0x7fe4c4160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59762/0 (socket says 172.21.15.52:59762) 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4ca4e3640 1 -- 172.21.15.52:0/2590227010 learned_addr learned my addr 172.21.15.52:0/2590227010 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4c9ce2640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c40d2ea0 0x7fe4c4161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.569+0000 7fe4c94e1640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c40d3330 0x7fe4c41f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4ca4e3640 1 -- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c40d3330 msgr2=0x7fe4c41f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4ca4e3640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c40d3330 0x7fe4c41f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4ca4e3640 1 -- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c40d2ea0 msgr2=0x7fe4c4161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4ca4e3640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c40d2ea0 0x7fe4c4161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4ca4e3640 1 -- 172.21.15.52:0/2590227010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c41f5210 con 0x7fe4c415f2d0 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4ca4e3640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c415f2d0 0x7fe4c4160590 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009eb0 tx=0x7fe4bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4ba7fc640 1 -- 172.21.15.52:0/2590227010 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc040070 con 0x7fe4c415f2d0 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4ba7fc640 1 -- 172.21.15.52:0/2590227010 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4bc031c40 con 0x7fe4c415f2d0 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4ba7fc640 1 -- 172.21.15.52:0/2590227010 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc003990 con 0x7fe4c415f2d0 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c4159e90 con 0x7fe4c415f2d0 2024-05-11T01:13:59.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.570+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c415fab0 con 0x7fe4c415f2d0 2024-05-11T01:13:59.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.571+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c4069780 con 0x7fe4c415f2d0 2024-05-11T01:13:59.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.572+0000 7fe4ba7fc640 1 -- 172.21.15.52:0/2590227010 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fe4bc039040 con 0x7fe4c415f2d0 2024-05-11T01:13:59.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.572+0000 7fe4ba7fc640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fe4a80762d0 0x7fe4a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.573+0000 7fe4ba7fc640 1 -- 172.21.15.52:0/2590227010 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6265+0+0 (secure 0 0 0) 0x7fe4bc0b8890 con 0x7fe4c415f2d0 2024-05-11T01:13:59.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.573+0000 7fe4c9ce2640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fe4a80762d0 0x7fe4a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.573+0000 7fe4c9ce2640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fe4a80762d0 0x7fe4a8078790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0006fd0 tx=0x7fe4c0009060 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:59.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.577+0000 7fe4ba7fc640 1 -- 172.21.15.52:0/2590227010 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe4bc0818d0 con 0x7fe4c415f2d0 2024-05-11T01:13:59.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.769+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4c40b1c40 con 0x7fe4c415f2d0 2024-05-11T01:13:59.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.771+0000 7fe4ba7fc640 1 -- 172.21.15.52:0/2590227010 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+17467 (secure 0 0 0) 0x7fe4bc085bd0 con 0x7fe4c415f2d0 2024-05-11T01:13:59.770 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:13:59.770 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":60,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:13:58.799389+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:13:56.478344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-11T01:13:57.758014+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":9,"length":1},{"begin":13,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":46,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":57,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":67,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":77,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":94,"length":2},{"begin":99,"length":1},{"begin":103,"length":1},{"begin":106,"length":1},{"begin":110,"length":1},{"begin":112,"length":1},{"begin":114,"length":1},{"begin":194,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.772+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fe4a80762d0 msgr2=0x7fe4a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.772+0000 7fe4cbf46640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fe4a80762d0 0x7fe4a8078790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0006fd0 tx=0x7fe4c0009060 comp rx=0 tx=0).stop 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.772+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c415f2d0 msgr2=0x7fe4c4160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.772+0000 7fe4cbf46640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c415f2d0 0x7fe4c4160590 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009eb0 tx=0x7fe4bc004a40 comp rx=0 tx=0).stop 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.773+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 shutdown_connections 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.773+0000 7fe4cbf46640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fe4a80762d0 0x7fe4a8078790 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.773+0000 7fe4cbf46640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4c415f2d0 0x7fe4c4160590 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.773+0000 7fe4cbf46640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c40d3330 0x7fe4c41f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.773+0000 7fe4cbf46640 1 --2- 172.21.15.52:0/2590227010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c40d2ea0 0x7fe4c4161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.773+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 >> 172.21.15.52:0/2590227010 conn(0x7fe4c409ba20 msgr2=0x7fe4c40703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:59.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.773+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 shutdown_connections 2024-05-11T01:13:59.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.773+0000 7fe4cbf46640 1 -- 172.21.15.52:0/2590227010 wait complete. 2024-05-11T01:13:59.783 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:13:59.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.888+0000 7f0446ff0640 1 Processor -- start 2024-05-11T01:13:59.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.888+0000 7f0446ff0640 1 -- start start 2024-05-11T01:13:59.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.888+0000 7f0446ff0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.888+0000 7f0446ff0640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04400d2970 con 0x7f04401602b0 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.888+0000 7f0446ff0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0440163250 con 0x7f04400d3d90 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.888+0000 7f0446ff0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f0440163530 con 0x7f044015f430 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f043ffff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f04401602b0 0x7f04400d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:48008/0 (socket says 172.21.15.52:48008) 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f043ffff640 1 -- 172.21.15.52:0/1540504402 learned_addr learned my addr 172.21.15.52:0/1540504402 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f0444d8c640 1 --2- 172.21.15.52:0/1540504402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f043f7fe640 1 -- 172.21.15.52:0/1540504402 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073703387 0 0) 0x7f04400d2970 con 0x7f04401602b0 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f043f7fe640 1 -- 172.21.15.52:0/1540504402 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0430003520 con 0x7f04401602b0 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f043f7fe640 1 -- 172.21.15.52:0/1540504402 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3868756102 0 0) 0x7f0440163250 con 0x7f04400d3d90 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f043f7fe640 1 -- 172.21.15.52:0/1540504402 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04400d2970 con 0x7f04400d3d90 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f0444d8c640 1 -- 172.21.15.52:0/1540504402 >> v1:172.21.15.119:6789/0 conn(0x7f04400d3d90 legacy=0x7f04400d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f0444d8c640 1 -- 172.21.15.52:0/1540504402 >> v1:172.21.15.52:6789/0 conn(0x7f04401602b0 legacy=0x7f04400d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.889+0000 7f0444d8c640 1 -- 172.21.15.52:0/1540504402 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0440163810 con 0x7f044015f430 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.890+0000 7f0444d8c640 1 --2- 172.21.15.52:0/1540504402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044015fd70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f04340097d0 tx=0x7f043402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f69df77a1f71db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.890+0000 7f043f7fe640 1 -- 172.21.15.52:0/1540504402 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434037070 con 0x7f044015f430 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.890+0000 7f043f7fe640 1 -- 172.21.15.52:0/1540504402 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0434032440 con 0x7f044015f430 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.890+0000 7f043f7fe640 1 -- 172.21.15.52:0/1540504402 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04340327f0 con 0x7f044015f430 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.891+0000 7f0446ff0640 1 -- 172.21.15.52:0/1540504402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 msgr2=0x7f044015fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.891+0000 7f0446ff0640 1 --2- 172.21.15.52:0/1540504402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044015fd70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f04340097d0 tx=0x7f043402a210 comp rx=0 tx=0).stop 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.891+0000 7f0446ff0640 1 -- 172.21.15.52:0/1540504402 shutdown_connections 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.891+0000 7f0446ff0640 1 --2- 172.21.15.52:0/1540504402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044015fd70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.891+0000 7f0446ff0640 1 -- 172.21.15.52:0/1540504402 >> 172.21.15.52:0/1540504402 conn(0x7f044008eb70 msgr2=0x7f044009e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:13:59.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.891+0000 7f0446ff0640 1 -- 172.21.15.52:0/1540504402 shutdown_connections 2024-05-11T01:13:59.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.891+0000 7f0446ff0640 1 -- 172.21.15.52:0/1540504402 wait complete. 2024-05-11T01:13:59.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.892+0000 7f0446ff0640 1 Processor -- start 2024-05-11T01:13:59.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.892+0000 7f0446ff0640 1 -- start start 2024-05-11T01:13:59.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0446ff0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04400d3d90 0x7f04400c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0446ff0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044014acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0446ff0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04401602b0 0x7f0440161600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0446ff0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0440162eb0 con 0x7f04400d3d90 2024-05-11T01:13:59.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0446ff0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f044015f810 con 0x7f04401602b0 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0446ff0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f0440163030 con 0x7f044015f430 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0444d8c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04400d3d90 0x7f04400c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0444d8c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04400d3d90 0x7f04400c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59772/0 (socket says 172.21.15.52:59772) 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0444d8c640 1 -- 172.21.15.52:0/3183526529 learned_addr learned my addr 172.21.15.52:0/3183526529 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f044558d640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04401602b0 0x7f0440161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f043ffff640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044014acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0444d8c640 1 -- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 msgr2=0x7f044014acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0444d8c640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044014acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0444d8c640 1 -- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04401602b0 msgr2=0x7f0440161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0444d8c640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04401602b0 0x7f0440161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f0444d8c640 1 -- 172.21.15.52:0/3183526529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04400ca770 con 0x7f04400d3d90 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.893+0000 7f043ffff640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044014acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:13:59.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.894+0000 7f0444d8c640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04400d3d90 0x7f04400c8ce0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0434035040 tx=0x7f04340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:59.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.894+0000 7f043d7fa640 1 -- 172.21.15.52:0/3183526529 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434037040 con 0x7f04400d3d90 2024-05-11T01:13:59.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.894+0000 7f043d7fa640 1 -- 172.21.15.52:0/3183526529 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04340044c0 con 0x7f04400d3d90 2024-05-11T01:13:59.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.894+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044014b230 con 0x7f04400d3d90 2024-05-11T01:13:59.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.894+0000 7f043d7fa640 1 -- 172.21.15.52:0/3183526529 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04340325f0 con 0x7f04400d3d90 2024-05-11T01:13:59.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.894+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04400cafd0 con 0x7f04400d3d90 2024-05-11T01:13:59.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.895+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0440069780 con 0x7f04400d3d90 2024-05-11T01:13:59.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.895+0000 7f043d7fa640 1 -- 172.21.15.52:0/3183526529 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f0434044020 con 0x7f04400d3d90 2024-05-11T01:13:59.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.896+0000 7f043d7fa640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f04240762d0 0x7f0424078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:13:59.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.896+0000 7f043d7fa640 1 -- 172.21.15.52:0/3183526529 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6265+0+0 (secure 0 0 0) 0x7f04340b7910 con 0x7f04400d3d90 2024-05-11T01:13:59.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.896+0000 7f043ffff640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f04240762d0 0x7f0424078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:13:59.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.897+0000 7f043ffff640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f04240762d0 0x7f0424078790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f042c001220 tx=0x7f042c009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:13:59.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:13:59.901+0000 7f043d7fa640 1 -- 172.21.15.52:0/3183526529 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f043404b020 con 0x7f04400d3d90 2024-05-11T01:14:00.054 INFO:tasks.ceph.mds.a.smithi052.stderr:2024-05-11T01:14:00.055+0000 7f8ebe41b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T01:14:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.096+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04400b5d20 con 0x7f04400d3d90 2024-05-11T01:14:00.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.097+0000 7f043d7fa640 1 -- 172.21.15.52:0/3183526529 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+17467 (secure 0 0 0) 0x7f0434080fb0 con 0x7f04400d3d90 2024-05-11T01:14:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:14:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":60,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:13:58.799389+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:13:56.478344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-11T01:13:57.758014+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":9,"length":1},{"begin":13,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":46,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":57,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":67,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":77,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":94,"length":2},{"begin":99,"length":1},{"begin":103,"length":1},{"begin":106,"length":1},{"begin":110,"length":1},{"begin":112,"length":1},{"begin":114,"length":1},{"begin":194,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:14:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f04240762d0 msgr2=0x7f0424078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f04240762d0 0x7f0424078790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f042c001220 tx=0x7f042c009040 comp rx=0 tx=0).stop 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04400d3d90 msgr2=0x7f04400c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04400d3d90 0x7f04400c8ce0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0434035040 tx=0x7f04340046c0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 shutdown_connections 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f04240762d0 0x7f0424078790 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04401602b0 0x7f0440161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f044015f430 0x7f044014acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 --2- 172.21.15.52:0/3183526529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04400d3d90 0x7f04400c8ce0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 >> 172.21.15.52:0/3183526529 conn(0x7f044008eb70 msgr2=0x7f044006d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 shutdown_connections 2024-05-11T01:14:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.099+0000 7f0446ff0640 1 -- 172.21.15.52:0/3183526529 wait complete. 2024-05-11T01:14:00.110 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T01:14:00.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.217+0000 7f5628083640 1 Processor -- start 2024-05-11T01:14:00.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.217+0000 7f5628083640 1 -- start start 2024-05-11T01:14:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5628083640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56200ce2c0 0x7f56200c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5628083640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f56200c8ec0 0x7f56200cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5628083640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56200d09c0 con 0x7f56200cad20 2024-05-11T01:14:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5628083640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f56200cb100 con 0x7f56200c8ec0 2024-05-11T01:14:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5628083640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f56200d0c60 con 0x7f56200ce2c0 2024-05-11T01:14:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5626620640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f56200c8ec0 0x7f56200cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:00.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5625e1f640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f56200cad20 0x7f56200c8270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:48014/0 (socket says 172.21.15.52:48014) 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5625e1f640 1 -- 172.21.15.52:0/4252480043 learned_addr learned my addr 172.21.15.52:0/4252480043 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f562561e640 1 --2- 172.21.15.52:0/4252480043 >> v2:172.21.15.119:3300/0 conn(0x7f56200ce2c0 0x7f56200c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5626620640 1 -- 172.21.15.52:0/4252480043 >> v2:172.21.15.119:3300/0 conn(0x7f56200ce2c0 msgr2=0x7f56200c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5626620640 1 --2- 172.21.15.52:0/4252480043 >> v2:172.21.15.119:3300/0 conn(0x7f56200ce2c0 0x7f56200c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5626620640 1 -- 172.21.15.52:0/4252480043 >> v1:172.21.15.52:6789/0 conn(0x7f56200cad20 legacy=0x7f56200c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5624e1d640 1 -- 172.21.15.52:0/4252480043 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522905845 0 0) 0x7f56200d09c0 con 0x7f56200cad20 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.218+0000 7f5626620640 1 -- 172.21.15.52:0/4252480043 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56200d0f40 con 0x7f56200c8ec0 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.219+0000 7f5626620640 1 --2- 172.21.15.52:0/4252480043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200c8ec0 0x7f56200cc800 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5620073700 tx=0x7f561802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75b87aab6e5363c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.219+0000 7f5624e1d640 1 -- 172.21.15.52:0/4252480043 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618007740 con 0x7f56200c8ec0 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.219+0000 7f5624e1d640 1 -- 172.21.15.52:0/4252480043 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56180078e0 con 0x7f56200c8ec0 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.219+0000 7f5624e1d640 1 -- 172.21.15.52:0/4252480043 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618007cb0 con 0x7f56200c8ec0 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.220+0000 7f5628083640 1 -- 172.21.15.52:0/4252480043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200c8ec0 msgr2=0x7f56200cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.220+0000 7f5628083640 1 --2- 172.21.15.52:0/4252480043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200c8ec0 0x7f56200cc800 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5620073700 tx=0x7f561802a680 comp rx=0 tx=0).stop 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.220+0000 7f5628083640 1 -- 172.21.15.52:0/4252480043 shutdown_connections 2024-05-11T01:14:00.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.220+0000 7f5628083640 1 --2- 172.21.15.52:0/4252480043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200c8ec0 0x7f56200cc800 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.220+0000 7f5628083640 1 --2- 172.21.15.52:0/4252480043 >> v2:172.21.15.119:3300/0 conn(0x7f56200ce2c0 0x7f56200c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.220+0000 7f5628083640 1 -- 172.21.15.52:0/4252480043 >> 172.21.15.52:0/4252480043 conn(0x7f56200b2d30 msgr2=0x7f56200b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.220+0000 7f5628083640 1 -- 172.21.15.52:0/4252480043 shutdown_connections 2024-05-11T01:14:00.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.220+0000 7f5628083640 1 -- 172.21.15.52:0/4252480043 wait complete. 2024-05-11T01:14:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.221+0000 7f5628083640 1 Processor -- start 2024-05-11T01:14:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.221+0000 7f5628083640 1 -- start start 2024-05-11T01:14:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f5628083640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cad20 0x7f562015a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f5628083640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200ce2c0 0x7f56201f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f5628083640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5620158690 0x7f56200b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f5628083640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56200ceb50 con 0x7f56200ce2c0 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f5628083640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56200ce9d0 con 0x7f56200cad20 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f5628083640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f56200ced70 con 0x7f5620158690 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f562561e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200ce2c0 0x7f56201f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f562561e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200ce2c0 0x7f56201f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59798/0 (socket says 172.21.15.52:59798) 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f562561e640 1 -- 172.21.15.52:0/4143094253 learned_addr learned my addr 172.21.15.52:0/4143094253 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f5625e1f640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cad20 0x7f562015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f5626620640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5620158690 0x7f56200b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f562561e640 1 -- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5620158690 msgr2=0x7f56200b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f562561e640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5620158690 0x7f56200b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f562561e640 1 -- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cad20 msgr2=0x7f562015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f562561e640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cad20 0x7f562015a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f562561e640 1 -- 172.21.15.52:0/4143094253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56200c4da0 con 0x7f56200ce2c0 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.222+0000 7f5625e1f640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cad20 0x7f562015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:14:00.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.223+0000 7f562561e640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200ce2c0 0x7f56201f4c90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f561000d5d0 tx=0x7f561000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.223+0000 7f56167fc640 1 -- 172.21.15.52:0/4143094253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610004240 con 0x7f56200ce2c0 2024-05-11T01:14:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.223+0000 7f56167fc640 1 -- 172.21.15.52:0/4143094253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56100043e0 con 0x7f56200ce2c0 2024-05-11T01:14:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.223+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56200c5080 con 0x7f56200ce2c0 2024-05-11T01:14:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.223+0000 7f56167fc640 1 -- 172.21.15.52:0/4143094253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610003830 con 0x7f56200ce2c0 2024-05-11T01:14:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.223+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56200c5640 con 0x7f56200ce2c0 2024-05-11T01:14:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.224+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56200695d0 con 0x7f56200ce2c0 2024-05-11T01:14:00.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.224+0000 7f56167fc640 1 -- 172.21.15.52:0/4143094253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f56100039d0 con 0x7f56200ce2c0 2024-05-11T01:14:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.225+0000 7f56167fc640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56040762d0 0x7f5604078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.225+0000 7f56167fc640 1 -- 172.21.15.52:0/4143094253 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6265+0+0 (secure 0 0 0) 0x7f5610097b40 con 0x7f56200ce2c0 2024-05-11T01:14:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.225+0000 7f5625e1f640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56040762d0 0x7f5604078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:00.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.226+0000 7f5625e1f640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56040762d0 0x7f5604078790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f561c000f60 tx=0x7f561c007bb0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:00.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.229+0000 7f56167fc640 1 -- 172.21.15.52:0/4143094253 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5610061270 con 0x7f56200ce2c0 2024-05-11T01:14:00.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:00.464+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f56200b3b80 con 0x7f56200ce2c0 2024-05-11T01:14:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.434+0000 7f56167fc640 1 -- 172.21.15.52:0/4143094253 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f5610060b70 con 0x7f56200ce2c0 2024-05-11T01:14:01.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56040762d0 msgr2=0x7f5604078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:01.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56040762d0 0x7f5604078790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f561c000f60 tx=0x7f561c007bb0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200ce2c0 msgr2=0x7f56201f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:01.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200ce2c0 0x7f56201f4c90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f561000d5d0 tx=0x7f561000da90 comp rx=0 tx=0).stop 2024-05-11T01:14:01.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 shutdown_connections 2024-05-11T01:14:01.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f56040762d0 0x7f5604078790 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5620158690 0x7f56200b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56200ce2c0 0x7f56201f4c90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 --2- 172.21.15.52:0/4143094253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cad20 0x7f562015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 >> 172.21.15.52:0/4143094253 conn(0x7f56200b2d30 msgr2=0x7f562006e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:01.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 shutdown_connections 2024-05-11T01:14:01.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.437+0000 7f5628083640 1 -- 172.21.15.52:0/4143094253 wait complete. 2024-05-11T01:14:01.450 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T01:14:01.450 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T01:14:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.565+0000 7f725cecc640 1 Processor -- start 2024-05-11T01:14:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.565+0000 7f725cecc640 1 -- start start 2024-05-11T01:14:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.565+0000 7f725cecc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f725815f2f0 0x7f72580cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.565+0000 7f725cecc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f72580d2d60 0x7f72580d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.565+0000 7f725cecc640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72580d3a90 con 0x7f72580d4e40 2024-05-11T01:14:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.565+0000 7f725cecc640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f72580d3d70 con 0x7f72580d2d60 2024-05-11T01:14:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.565+0000 7f725cecc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f72580d4050 con 0x7f725815f2f0 2024-05-11T01:14:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.566+0000 7f7256d9d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f72580d2d60 0x7f72580d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.566+0000 7f724ffff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f72580d4e40 0x7f72580d5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:48022/0 (socket says 172.21.15.52:48022) 2024-05-11T01:14:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.566+0000 7f724ffff640 1 -- 172.21.15.52:0/264132087 learned_addr learned my addr 172.21.15.52:0/264132087 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:14:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.566+0000 7f725659c640 1 --2- 172.21.15.52:0/264132087 >> v2:172.21.15.119:3300/0 conn(0x7f725815f2f0 0x7f72580cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.566+0000 7f7256d9d640 1 -- 172.21.15.52:0/264132087 >> v2:172.21.15.119:3300/0 conn(0x7f725815f2f0 msgr2=0x7f72580cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.566+0000 7f7256d9d640 1 --2- 172.21.15.52:0/264132087 >> v2:172.21.15.119:3300/0 conn(0x7f725815f2f0 0x7f72580cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.566+0000 7f7256d9d640 1 -- 172.21.15.52:0/264132087 >> v1:172.21.15.52:6789/0 conn(0x7f72580d4e40 legacy=0x7f72580d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.566+0000 7f7256d9d640 1 -- 172.21.15.52:0/264132087 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72580b4e60 con 0x7f72580d2d60 2024-05-11T01:14:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f7256d9d640 1 --2- 172.21.15.52:0/264132087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 0x7f72580d31e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f72580d5fd0 tx=0x7f725002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcd43b31afb8332 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f7255d9b640 1 -- 172.21.15.52:0/264132087 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250007740 con 0x7f72580d2d60 2024-05-11T01:14:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f7255d9b640 1 -- 172.21.15.52:0/264132087 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72500078e0 con 0x7f72580d2d60 2024-05-11T01:14:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f7255d9b640 1 -- 172.21.15.52:0/264132087 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250007cb0 con 0x7f72580d2d60 2024-05-11T01:14:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f725cecc640 1 -- 172.21.15.52:0/264132087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 msgr2=0x7f72580d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f725cecc640 1 --2- 172.21.15.52:0/264132087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 0x7f72580d31e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f72580d5fd0 tx=0x7f725002a680 comp rx=0 tx=0).stop 2024-05-11T01:14:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f725cecc640 1 -- 172.21.15.52:0/264132087 shutdown_connections 2024-05-11T01:14:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f725cecc640 1 --2- 172.21.15.52:0/264132087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 0x7f72580d31e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f725cecc640 1 --2- 172.21.15.52:0/264132087 >> v2:172.21.15.119:3300/0 conn(0x7f725815f2f0 0x7f72580cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f725cecc640 1 -- 172.21.15.52:0/264132087 >> 172.21.15.52:0/264132087 conn(0x7f725808ebb0 msgr2=0x7f725809e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f725cecc640 1 -- 172.21.15.52:0/264132087 shutdown_connections 2024-05-11T01:14:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.567+0000 7f725cecc640 1 -- 172.21.15.52:0/264132087 wait complete. 2024-05-11T01:14:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.569+0000 7f725cecc640 1 Processor -- start 2024-05-11T01:14:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.569+0000 7f725cecc640 1 -- start start 2024-05-11T01:14:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725cecc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 0x7f725815af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725cecc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72580d4e40 0x7f725815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725cecc640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f725815f2f0 0x7f725809ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725cecc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f72580d01e0 con 0x7f72580d2d60 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725cecc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f72580d3720 con 0x7f72580d4e40 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725cecc640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f72580d0360 con 0x7f725815f2f0 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 0x7f725815af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 0x7f725815af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59814/0 (socket says 172.21.15.52:59814) 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725659c640 1 -- 172.21.15.52:0/2657223816 learned_addr learned my addr 172.21.15.52:0/2657223816 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f724ffff640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72580d4e40 0x7f725815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f7256d9d640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f725815f2f0 0x7f725809ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725659c640 1 -- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f725815f2f0 msgr2=0x7f725809ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725659c640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f725815f2f0 0x7f725809ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725659c640 1 -- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72580d4e40 msgr2=0x7f725815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725659c640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72580d4e40 0x7f725815b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725659c640 1 -- 172.21.15.52:0/2657223816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725815ba80 con 0x7f72580d2d60 2024-05-11T01:14:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f725659c640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 0x7f725815af30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f724800d800 tx=0x7f724800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.570+0000 7f724effd640 1 -- 172.21.15.52:0/2657223816 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724800b6c0 con 0x7f72580d2d60 2024-05-11T01:14:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.571+0000 7f724effd640 1 -- 172.21.15.52:0/2657223816 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7248019070 con 0x7f72580d2d60 2024-05-11T01:14:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.571+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725815bd60 con 0x7f72580d2d60 2024-05-11T01:14:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.571+0000 7f724effd640 1 -- 172.21.15.52:0/2657223816 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724800ba80 con 0x7f72580d2d60 2024-05-11T01:14:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.571+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725815c320 con 0x7f72580d2d60 2024-05-11T01:14:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.572+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72580695d0 con 0x7f72580d2d60 2024-05-11T01:14:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.572+0000 7f724effd640 1 -- 172.21.15.52:0/2657223816 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f724800bc20 con 0x7f72580d2d60 2024-05-11T01:14:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.573+0000 7f724effd640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7238076300 0x7f72380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.573+0000 7f724effd640 1 -- 172.21.15.52:0/2657223816 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f7248097b80 con 0x7f72580d2d60 2024-05-11T01:14:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.573+0000 7f724ffff640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7238076300 0x7f72380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.574+0000 7f724ffff640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7238076300 0x7f72380787c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f725815cfd0 tx=0x7f7240009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.577+0000 7f724effd640 1 -- 172.21.15.52:0/2657223816 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7248061340 con 0x7f72580d2d60 2024-05-11T01:14:01.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:01.809+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f72580d24f0 con 0x7f72580d2d60 2024-05-11T01:14:02.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.443+0000 7f724effd640 1 -- 172.21.15.52:0/2657223816 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f7248060c40 con 0x7f72580d2d60 2024-05-11T01:14:02.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7238076300 msgr2=0x7f72380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7238076300 0x7f72380787c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f725815cfd0 tx=0x7f7240009040 comp rx=0 tx=0).stop 2024-05-11T01:14:02.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 msgr2=0x7f725815af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 0x7f725815af30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f724800d800 tx=0x7f724800dcc0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 shutdown_connections 2024-05-11T01:14:02.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7238076300 0x7f72380787c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f725815f2f0 0x7f725809ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72580d4e40 0x7f725815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 --2- 172.21.15.52:0/2657223816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72580d2d60 0x7f725815af30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 >> 172.21.15.52:0/2657223816 conn(0x7f725808ebb0 msgr2=0x7f7258161df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:02.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 shutdown_connections 2024-05-11T01:14:02.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.446+0000 7f725cecc640 1 -- 172.21.15.52:0/2657223816 wait complete. 2024-05-11T01:14:02.461 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T01:14:02.462 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:14:02.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.572+0000 7f8be9ad1640 1 Processor -- start 2024-05-11T01:14:02.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.572+0000 7f8be9ad1640 1 -- start start 2024-05-11T01:14:02.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.573+0000 7f8be9ad1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8be41576a0 0x7f8be40add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.573+0000 7f8be9ad1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8be40ae290 0x7f8be40af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.573+0000 7f8be9ad1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be40743e0 con 0x7f8be415e1d0 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.573+0000 7f8be9ad1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8be4160c50 con 0x7f8be40ae290 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.573+0000 7f8be9ad1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8be4160f30 con 0x7f8be41576a0 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.573+0000 7f8be2ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8be40ae290 0x7f8be40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.573+0000 7f8be2ffd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8be40ae290 0x7f8be40af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49512/0 (socket says 172.21.15.52:49512) 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.573+0000 7f8be2ffd640 1 -- 172.21.15.52:0/422443581 learned_addr learned my addr 172.21.15.52:0/422443581 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.574+0000 7f8be2ffd640 1 -- 172.21.15.52:0/422443581 >> v2:172.21.15.119:3300/0 conn(0x7f8be41576a0 msgr2=0x7f8be40add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.574+0000 7f8be2ffd640 1 --2- 172.21.15.52:0/422443581 >> v2:172.21.15.119:3300/0 conn(0x7f8be41576a0 0x7f8be40add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.574+0000 7f8be2ffd640 1 -- 172.21.15.52:0/422443581 >> v1:172.21.15.119:6789/0 conn(0x7f8be415e1d0 legacy=0x7f8be40721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:02.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.574+0000 7f8be2ffd640 1 -- 172.21.15.52:0/422443581 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be4161210 con 0x7f8be40ae290 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.574+0000 7f8be2ffd640 1 --2- 172.21.15.52:0/422443581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be40ae290 0x7f8be40af950 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8be40d1e20 tx=0x7f8bd802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43dae4c02055abb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.574+0000 7f8be27fc640 1 -- 172.21.15.52:0/422443581 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8037070 con 0x7f8be40ae290 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be27fc640 1 -- 172.21.15.52:0/422443581 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd8032440 con 0x7f8be40ae290 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be27fc640 1 -- 172.21.15.52:0/422443581 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8032810 con 0x7f8be40ae290 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be9ad1640 1 -- 172.21.15.52:0/422443581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be40ae290 msgr2=0x7f8be40af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be9ad1640 1 --2- 172.21.15.52:0/422443581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be40ae290 0x7f8be40af950 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8be40d1e20 tx=0x7f8bd802a1c0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be9ad1640 1 -- 172.21.15.52:0/422443581 shutdown_connections 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be9ad1640 1 --2- 172.21.15.52:0/422443581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be40ae290 0x7f8be40af950 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be9ad1640 1 --2- 172.21.15.52:0/422443581 >> v2:172.21.15.119:3300/0 conn(0x7f8be41576a0 0x7f8be40add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be9ad1640 1 -- 172.21.15.52:0/422443581 >> 172.21.15.52:0/422443581 conn(0x7f8be40b2a00 msgr2=0x7f8be40b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be9ad1640 1 -- 172.21.15.52:0/422443581 shutdown_connections 2024-05-11T01:14:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.575+0000 7f8be9ad1640 1 -- 172.21.15.52:0/422443581 wait complete. 2024-05-11T01:14:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.576+0000 7f8be9ad1640 1 Processor -- start 2024-05-11T01:14:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.577+0000 7f8be9ad1640 1 -- start start 2024-05-11T01:14:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.577+0000 7f8be9ad1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be40ae290 0x7f8be4142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.577+0000 7f8be9ad1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be41576a0 0x7f8be4151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.577+0000 7f8be9ad1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8be415e1d0 0x7f8be415f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.577+0000 7f8be9ad1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8be40afe90 con 0x7f8be41576a0 2024-05-11T01:14:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.577+0000 7f8be9ad1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8be406e050 con 0x7f8be40ae290 2024-05-11T01:14:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.577+0000 7f8be9ad1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8be40b0010 con 0x7f8be415e1d0 2024-05-11T01:14:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.577+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be41576a0 0x7f8be4151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.577+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be41576a0 0x7f8be4151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49524/0 (socket says 172.21.15.52:49524) 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be2ffd640 1 -- 172.21.15.52:0/661123093 learned_addr learned my addr 172.21.15.52:0/661123093 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be37fe640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be40ae290 0x7f8be4142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be3fff640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8be415e1d0 0x7f8be415f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be2ffd640 1 -- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8be415e1d0 msgr2=0x7f8be415f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be2ffd640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8be415e1d0 0x7f8be415f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be2ffd640 1 -- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be40ae290 msgr2=0x7f8be4142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be2ffd640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be40ae290 0x7f8be4142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be2ffd640 1 -- 172.21.15.52:0/661123093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be414d160 con 0x7f8be41576a0 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be37fe640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be40ae290 0x7f8be4142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be3fff640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8be415e1d0 0x7f8be415f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be2ffd640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be41576a0 0x7f8be4151390 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8000c00 tx=0x7f8bd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8bcbfff640 1 -- 172.21.15.52:0/661123093 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8041070 con 0x7f8be41576a0 2024-05-11T01:14:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8bcbfff640 1 -- 172.21.15.52:0/661123093 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd803c870 con 0x7f8be41576a0 2024-05-11T01:14:02.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be41f6450 con 0x7f8be41576a0 2024-05-11T01:14:02.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8bcbfff640 1 -- 172.21.15.52:0/661123093 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8037040 con 0x7f8be41576a0 2024-05-11T01:14:02.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.578+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be41f6a10 con 0x7f8be41576a0 2024-05-11T01:14:02.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.579+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be4069780 con 0x7f8be41576a0 2024-05-11T01:14:02.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.580+0000 7f8bcbfff640 1 -- 172.21.15.52:0/661123093 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f8bd8044070 con 0x7f8be41576a0 2024-05-11T01:14:02.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.580+0000 7f8bcbfff640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8bd00762d0 0x7f8bd0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.581+0000 7f8bcbfff640 1 -- 172.21.15.52:0/661123093 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f8bd80b8370 con 0x7f8be41576a0 2024-05-11T01:14:02.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.581+0000 7f8be37fe640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8bd00762d0 0x7f8bd0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:02.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.581+0000 7f8be37fe640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8bd00762d0 0x7f8bd0078790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4006fd0 tx=0x7f8bd4008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:02.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.585+0000 7f8bcbfff640 1 -- 172.21.15.52:0/661123093 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8bd8081a90 con 0x7f8be41576a0 2024-05-11T01:14:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.815+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8be406efc0 con 0x7f8be41576a0 2024-05-11T01:14:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.816+0000 7f8bcbfff640 1 -- 172.21.15.52:0/661123093 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5178 (secure 0 0 0) 0x7f8bd8081390 con 0x7f8be41576a0 2024-05-11T01:14:02.815 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:14:02.815 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/4021589378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4021589378},{"type":"v1","addr":"172.21.15.119:6836","nonce":4021589378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/456790373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":456790373},{"type":"v1","addr":"172.21.15.119:6837","nonce":456790373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/1460397063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1460397063},{"type":"v1","addr":"172.21.15.52:6834","nonce":1460397063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:13:58.799356+0000","modified":"2024-05-11T01:14:01.810259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/59064433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":59064433},{"type":"v1","addr":"172.21.15.52:6835","nonce":59064433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5649,"qdb_cluster":[5649]},"id":3}]} 2024-05-11T01:14:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-05-11T01:14:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8bd00762d0 msgr2=0x7f8bd0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8bd00762d0 0x7f8bd0078790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4006fd0 tx=0x7f8bd4008040 comp rx=0 tx=0).stop 2024-05-11T01:14:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be41576a0 msgr2=0x7f8be4151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be41576a0 0x7f8be4151390 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8000c00 tx=0x7f8bd80046c0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 shutdown_connections 2024-05-11T01:14:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8bd00762d0 0x7f8bd0078790 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8be415e1d0 0x7f8be415f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be41576a0 0x7f8be4151390 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 --2- 172.21.15.52:0/661123093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be40ae290 0x7f8be4142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 >> 172.21.15.52:0/661123093 conn(0x7f8be40b2a00 msgr2=0x7f8be406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 shutdown_connections 2024-05-11T01:14:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.818+0000 7f8be9ad1640 1 -- 172.21.15.52:0/661123093 wait complete. 2024-05-11T01:14:02.828 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:14:02.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.933+0000 7f9f85587640 1 Processor -- start 2024-05-11T01:14:02.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.933+0000 7f9f85587640 1 -- start start 2024-05-11T01:14:02.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f85587640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9f8015f430 0x7f9f8015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f85587640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f801602b0 0x7f9f801606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f85587640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f80163250 con 0x7f9f800d1b50 2024-05-11T01:14:02.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f85587640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f9f80163530 con 0x7f9f8015f430 2024-05-11T01:14:02.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f85587640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f9f80163810 con 0x7f9f801602b0 2024-05-11T01:14:02.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f7effd640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f9f8015f430 0x7f9f8015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:02.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f7f7fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f9f800d1b50 0x7f9f800d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55208/0 (socket says 172.21.15.52:55208) 2024-05-11T01:14:02.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f7f7fe640 1 -- 172.21.15.52:0/201994114 learned_addr learned my addr 172.21.15.52:0/201994114 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:14:02.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f7effd640 1 -- 172.21.15.52:0/201994114 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f801602b0 msgr2=0x7f9f801606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:02.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f7effd640 1 --2- 172.21.15.52:0/201994114 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f801602b0 0x7f9f801606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f7effd640 1 -- 172.21.15.52:0/201994114 >> v1:172.21.15.52:6789/0 conn(0x7f9f800d1b50 legacy=0x7f9f800d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.934+0000 7f9f7effd640 1 -- 172.21.15.52:0/201994114 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f80163af0 con 0x7f9f8015f430 2024-05-11T01:14:02.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.935+0000 7f9f7effd640 1 --2- 172.21.15.52:0/201994114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f8015f430 0x7f9f8015fd70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9f700097a0 tx=0x7f9f7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ff8c7ec8c60338 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:02.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.935+0000 7f9f7e7fc640 1 -- 172.21.15.52:0/201994114 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70037070 con 0x7f9f8015f430 2024-05-11T01:14:02.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.935+0000 7f9f7e7fc640 1 -- 172.21.15.52:0/201994114 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f70032440 con 0x7f9f8015f430 2024-05-11T01:14:02.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.935+0000 7f9f7e7fc640 1 -- 172.21.15.52:0/201994114 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70032810 con 0x7f9f8015f430 2024-05-11T01:14:02.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.936+0000 7f9f85587640 1 -- 172.21.15.52:0/201994114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f8015f430 msgr2=0x7f9f8015fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.936+0000 7f9f85587640 1 --2- 172.21.15.52:0/201994114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f8015f430 0x7f9f8015fd70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9f700097a0 tx=0x7f9f7002a180 comp rx=0 tx=0).stop 2024-05-11T01:14:02.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.936+0000 7f9f85587640 1 -- 172.21.15.52:0/201994114 shutdown_connections 2024-05-11T01:14:02.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.936+0000 7f9f85587640 1 --2- 172.21.15.52:0/201994114 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f801602b0 0x7f9f801606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.936+0000 7f9f85587640 1 --2- 172.21.15.52:0/201994114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f8015f430 0x7f9f8015fd70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.936+0000 7f9f85587640 1 -- 172.21.15.52:0/201994114 >> 172.21.15.52:0/201994114 conn(0x7f9f8008eb70 msgr2=0x7f9f8009e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:02.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.936+0000 7f9f85587640 1 -- 172.21.15.52:0/201994114 shutdown_connections 2024-05-11T01:14:02.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.936+0000 7f9f85587640 1 -- 172.21.15.52:0/201994114 wait complete. 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.937+0000 7f9f85587640 1 Processor -- start 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.937+0000 7f9f85587640 1 -- start start 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f85587640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f800d1b50 0x7f9f8014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f85587640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f8015f430 0x7f9f800cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f85587640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f801602b0 0x7f9f800b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f85587640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80163030 con 0x7f9f800d1b50 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f85587640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80162eb0 con 0x7f9f8015f430 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f85587640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f9f800d2710 con 0x7f9f801602b0 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f800d1b50 0x7f9f8014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f800d1b50 0x7f9f8014b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49534/0 (socket says 172.21.15.52:49534) 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f7effd640 1 -- 172.21.15.52:0/170208759 learned_addr learned my addr 172.21.15.52:0/170208759 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f7effd640 1 -- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f801602b0 msgr2=0x7f9f800b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f7effd640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f801602b0 0x7f9f800b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f7effd640 1 -- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f8015f430 msgr2=0x7f9f800cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f77fff640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f8015f430 0x7f9f800cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f7effd640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f8015f430 0x7f9f800cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f7effd640 1 -- 172.21.15.52:0/170208759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f800cbaa0 con 0x7f9f800d1b50 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f7effd640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f800d1b50 0x7f9f8014b0f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9f70000c00 tx=0x7f9f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.938+0000 7f9f777fe640 1 -- 172.21.15.52:0/170208759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70037040 con 0x7f9f800d1b50 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.939+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f800cb7c0 con 0x7f9f800d1b50 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.939+0000 7f9f777fe640 1 -- 172.21.15.52:0/170208759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f70042070 con 0x7f9f800d1b50 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.939+0000 7f9f777fe640 1 -- 172.21.15.52:0/170208759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70030a70 con 0x7f9f800d1b50 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.939+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8014bde0 con 0x7f9f800d1b50 2024-05-11T01:14:02.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.940+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f80069780 con 0x7f9f800d1b50 2024-05-11T01:14:02.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.940+0000 7f9f777fe640 1 -- 172.21.15.52:0/170208759 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f9f70030c10 con 0x7f9f800d1b50 2024-05-11T01:14:02.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.941+0000 7f9f777fe640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f9f600762d0 0x7f9f60078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:02.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.941+0000 7f9f777fe640 1 -- 172.21.15.52:0/170208759 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f9f700b8360 con 0x7f9f800d1b50 2024-05-11T01:14:02.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.941+0000 7f9f77fff640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f9f600762d0 0x7f9f60078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:02.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.942+0000 7f9f77fff640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f9f600762d0 0x7f9f60078790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f800ce9c0 tx=0x7f9f68008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:02.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:02.945+0000 7f9f777fe640 1 -- 172.21.15.52:0/170208759 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9f70081980 con 0x7f9f800d1b50 2024-05-11T01:14:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.140+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f800d22c0 con 0x7f9f800d1b50 2024-05-11T01:14:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.141+0000 7f9f777fe640 1 -- 172.21.15.52:0/170208759 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+17454 (secure 0 0 0) 0x7f9f70081280 con 0x7f9f800d1b50 2024-05-11T01:14:03.140 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:14:03.140 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":61,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:14:00.418210+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:13:56.478344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-11T01:13:57.758014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":9,"length":1},{"begin":13,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":46,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":57,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":67,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":77,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":94,"length":2},{"begin":99,"length":1},{"begin":103,"length":1},{"begin":106,"length":1},{"begin":110,"length":1},{"begin":112,"length":1},{"begin":114,"length":1},{"begin":194,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:14:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f9f600762d0 msgr2=0x7f9f60078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f9f600762d0 0x7f9f60078790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f800ce9c0 tx=0x7f9f68008040 comp rx=0 tx=0).stop 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f800d1b50 msgr2=0x7f9f8014b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f800d1b50 0x7f9f8014b0f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9f70000c00 tx=0x7f9f700046c0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 shutdown_connections 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f9f600762d0 0x7f9f60078790 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9f801602b0 0x7f9f800b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f8015f430 0x7f9f800cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 --2- 172.21.15.52:0/170208759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f800d1b50 0x7f9f8014b0f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 >> 172.21.15.52:0/170208759 conn(0x7f9f8008eb70 msgr2=0x7f9f800b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 shutdown_connections 2024-05-11T01:14:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.143+0000 7f9f85587640 1 -- 172.21.15.52:0/170208759 wait complete. 2024-05-11T01:14:03.154 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T01:14:03.154 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:14:03.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.258+0000 7f8eba4a0640 1 Processor -- start 2024-05-11T01:14:03.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.258+0000 7f8eba4a0640 1 -- start start 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eba4a0640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8eb40d3330 0x7f8eb40b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eba4a0640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb40d1f90 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eba4a0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb40d2270 con 0x7f8eb415f2d0 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eba4a0640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb40d2550 con 0x7f8eb40d3330 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eb8a3d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8eb40d3330 0x7f8eb40b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eb8a3d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8eb40d3330 0x7f8eb40b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49538/0 (socket says 172.21.15.52:49538) 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eb37fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8eb40d2ea0 0x7f8eb40b2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55216/0 (socket says 172.21.15.52:55216) 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eb8a3d640 1 -- 172.21.15.52:0/1448892007 learned_addr learned my addr 172.21.15.52:0/1448892007 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eb8a3d640 1 -- 172.21.15.52:0/1448892007 >> v1:172.21.15.119:6789/0 conn(0x7f8eb415f2d0 legacy=0x7f8eb406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eb8a3d640 1 -- 172.21.15.52:0/1448892007 >> v1:172.21.15.52:6789/0 conn(0x7f8eb40d2ea0 legacy=0x7f8eb40b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eb2ffd640 1 -- 172.21.15.52:0/1448892007 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202216816 0 0) 0x7f8eb40d1f90 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.259+0000 7f8eb8a3d640 1 -- 172.21.15.52:0/1448892007 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb40d2830 con 0x7f8eb40d3330 2024-05-11T01:14:03.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.260+0000 7f8eb8a3d640 1 --2- 172.21.15.52:0/1448892007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d3330 0x7f8eb40b2d10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009e50 tx=0x7f8eac02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc45aa9312709e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.260+0000 7f8eb2ffd640 1 -- 172.21.15.52:0/1448892007 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac036070 con 0x7f8eb40d3330 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.260+0000 7f8eb2ffd640 1 -- 172.21.15.52:0/1448892007 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac031600 con 0x7f8eb40d3330 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.260+0000 7f8eb2ffd640 1 -- 172.21.15.52:0/1448892007 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0319d0 con 0x7f8eb40d3330 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.261+0000 7f8eba4a0640 1 -- 172.21.15.52:0/1448892007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d3330 msgr2=0x7f8eb40b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.261+0000 7f8eba4a0640 1 --2- 172.21.15.52:0/1448892007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d3330 0x7f8eb40b2d10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009e50 tx=0x7f8eac02a710 comp rx=0 tx=0).stop 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.261+0000 7f8eba4a0640 1 -- 172.21.15.52:0/1448892007 shutdown_connections 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.261+0000 7f8eba4a0640 1 --2- 172.21.15.52:0/1448892007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d3330 0x7f8eb40b2d10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.261+0000 7f8eba4a0640 1 -- 172.21.15.52:0/1448892007 >> 172.21.15.52:0/1448892007 conn(0x7f8eb409ba20 msgr2=0x7f8eb409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.261+0000 7f8eba4a0640 1 -- 172.21.15.52:0/1448892007 shutdown_connections 2024-05-11T01:14:03.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.261+0000 7f8eba4a0640 1 -- 172.21.15.52:0/1448892007 wait complete. 2024-05-11T01:14:03.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.262+0000 7f8eba4a0640 1 Processor -- start 2024-05-11T01:14:03.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.262+0000 7f8eba4a0640 1 -- start start 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eba4a0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d2ea0 0x7f8eb4162520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eba4a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40d3330 0x7f8eb41f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eba4a0640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8eb415f2d0 0x7f8eb4161430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eba4a0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb415f6b0 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eba4a0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb40d3740 con 0x7f8eb40d3330 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eba4a0640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb415f830 con 0x7f8eb415f2d0 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d2ea0 0x7f8eb4162520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d2ea0 0x7f8eb4162520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49552/0 (socket says 172.21.15.52:49552) 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb3fff640 1 -- 172.21.15.52:0/900272455 learned_addr learned my addr 172.21.15.52:0/900272455 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb37fe640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40d3330 0x7f8eb41f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb8a3d640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8eb415f2d0 0x7f8eb4161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb3fff640 1 -- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8eb415f2d0 msgr2=0x7f8eb4161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb3fff640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8eb415f2d0 0x7f8eb4161430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb3fff640 1 -- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40d3330 msgr2=0x7f8eb41f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb3fff640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40d3330 0x7f8eb41f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.263+0000 7f8eb3fff640 1 -- 172.21.15.52:0/900272455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb4160670 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.264+0000 7f8eb3fff640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d2ea0 0x7f8eb4162520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400b420 tx=0x7f8ea400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.264+0000 7f8eb0ff9640 1 -- 172.21.15.52:0/900272455 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4011830 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.264+0000 7f8eb0ff9640 1 -- 172.21.15.52:0/900272455 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea4016070 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.264+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb4160950 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.264+0000 7f8eb0ff9640 1 -- 172.21.15.52:0/900272455 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4011c10 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.264+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb41f5ed0 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.265+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb4069780 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.265+0000 7f8eb0ff9640 1 -- 172.21.15.52:0/900272455 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f8ea401b460 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.266+0000 7f8eb0ff9640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8ea00762d0 0x7f8ea0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.266+0000 7f8eb0ff9640 1 -- 172.21.15.52:0/900272455 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f8ea4097280 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.266+0000 7f8eb37fe640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8ea00762d0 0x7f8ea0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.267+0000 7f8eb37fe640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8ea00762d0 0x7f8ea0078790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8eb415d240 tx=0x7f8ea8007b50 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:03.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.270+0000 7f8eb0ff9640 1 -- 172.21.15.52:0/900272455 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ea4020020 con 0x7f8eb40d2ea0 2024-05-11T01:14:03.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.458+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8eb4160420 con 0x7f8ea00762d0 2024-05-11T01:14:03.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.465+0000 7f8eb0ff9640 1 -- 172.21.15.52:0/900272455 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250118 (secure 0 0 0) 0x7f8eb4160420 con 0x7f8ea00762d0 2024-05-11T01:14:03.467 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:14:03.468 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:14:03.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8ea00762d0 msgr2=0x7f8ea0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8ea00762d0 0x7f8ea0078790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8eb415d240 tx=0x7f8ea8007b50 comp rx=0 tx=0).stop 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d2ea0 msgr2=0x7f8eb4162520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d2ea0 0x7f8eb4162520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400b420 tx=0x7f8ea400b8e0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 shutdown_connections 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8ea00762d0 0x7f8ea0078790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8eb415f2d0 0x7f8eb4161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8eb40d3330 0x7f8eb41f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 --2- 172.21.15.52:0/900272455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8eb40d2ea0 0x7f8eb4162520 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 >> 172.21.15.52:0/900272455 conn(0x7f8eb409ba20 msgr2=0x7f8eb40b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 shutdown_connections 2024-05-11T01:14:03.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.468+0000 7f8eba4a0640 1 -- 172.21.15.52:0/900272455 wait complete. 2024-05-11T01:14:03.482 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-05-11T01:14:01.707404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":170,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":317360,"kb_used_data":6820,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754657360,"statfs":{"total":773094113280,"available":772769136640,"internally_reserved":0,"allocated":6983680,"data_stored":3917941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":317836849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.295286"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426499+0000","last_change":"2024-05-11T01:13:59.434893+0000","last_active":"2024-05-11T01:14:00.426499+0000","last_peered":"2024-05-11T01:14:00.426499+0000","last_clean":"2024-05-11T01:14:00.426499+0000","last_became_active":"2024-05-11T01:13:59.434594+0000","last_became_peered":"2024-05-11T01:13:59.434594+0000","last_unstale":"2024-05-11T01:14:00.426499+0000","last_undegraded":"2024-05-11T01:14:00.426499+0000","last_fullsized":"2024-05-11T01:14:00.426499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:49:33.685964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424479+0000","last_change":"2024-05-11T01:13:59.446282+0000","last_active":"2024-05-11T01:14:00.424479+0000","last_peered":"2024-05-11T01:14:00.424479+0000","last_clean":"2024-05-11T01:14:00.424479+0000","last_became_active":"2024-05-11T01:13:59.445971+0000","last_became_peered":"2024-05-11T01:13:59.445971+0000","last_unstale":"2024-05-11T01:14:00.424479+0000","last_undegraded":"2024-05-11T01:14:00.424479+0000","last_fullsized":"2024-05-11T01:14:00.424479+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:50.869559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423979+0000","last_change":"2024-05-11T01:13:59.430661+0000","last_active":"2024-05-11T01:14:00.423979+0000","last_peered":"2024-05-11T01:14:00.423979+0000","last_clean":"2024-05-11T01:14:00.423979+0000","last_became_active":"2024-05-11T01:13:59.430408+0000","last_became_peered":"2024-05-11T01:13:59.430408+0000","last_unstale":"2024-05-11T01:14:00.423979+0000","last_undegraded":"2024-05-11T01:14:00.423979+0000","last_fullsized":"2024-05-11T01:14:00.423979+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:14:03.379607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421742+0000","last_change":"2024-05-11T01:13:59.448424+0000","last_active":"2024-05-11T01:14:00.421742+0000","last_peered":"2024-05-11T01:14:00.421742+0000","last_clean":"2024-05-11T01:14:00.421742+0000","last_became_active":"2024-05-11T01:13:59.446986+0000","last_became_peered":"2024-05-11T01:13:59.446986+0000","last_unstale":"2024-05-11T01:14:00.421742+0000","last_undegraded":"2024-05-11T01:14:00.421742+0000","last_fullsized":"2024-05-11T01:14:00.421742+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424359+0000","last_change":"2024-05-11T01:13:59.446369+0000","last_active":"2024-05-11T01:14:00.424359+0000","last_peered":"2024-05-11T01:14:00.424359+0000","last_clean":"2024-05-11T01:14:00.424359+0000","last_became_active":"2024-05-11T01:13:59.445922+0000","last_became_peered":"2024-05-11T01:13:59.445922+0000","last_unstale":"2024-05-11T01:14:00.424359+0000","last_undegraded":"2024-05-11T01:14:00.424359+0000","last_fullsized":"2024-05-11T01:14:00.424359+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:15.278361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423902+0000","last_change":"2024-05-11T01:13:59.435535+0000","last_active":"2024-05-11T01:14:00.423902+0000","last_peered":"2024-05-11T01:14:00.423902+0000","last_clean":"2024-05-11T01:14:00.423902+0000","last_became_active":"2024-05-11T01:13:59.434634+0000","last_became_peered":"2024-05-11T01:13:59.434634+0000","last_unstale":"2024-05-11T01:14:00.423902+0000","last_undegraded":"2024-05-11T01:14:00.423902+0000","last_fullsized":"2024-05-11T01:14:00.423902+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:34:16.451030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425878+0000","last_change":"2024-05-11T01:13:59.434552+0000","last_active":"2024-05-11T01:14:00.425878+0000","last_peered":"2024-05-11T01:14:00.425878+0000","last_clean":"2024-05-11T01:14:00.425878+0000","last_became_active":"2024-05-11T01:13:59.433976+0000","last_became_peered":"2024-05-11T01:13:59.433976+0000","last_unstale":"2024-05-11T01:14:00.425878+0000","last_undegraded":"2024-05-11T01:14:00.425878+0000","last_fullsized":"2024-05-11T01:14:00.425878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421319+0000","last_change":"2024-05-11T01:13:59.447658+0000","last_active":"2024-05-11T01:14:00.421319+0000","last_peered":"2024-05-11T01:14:00.421319+0000","last_clean":"2024-05-11T01:14:00.421319+0000","last_became_active":"2024-05-11T01:13:59.446507+0000","last_became_peered":"2024-05-11T01:13:59.446507+0000","last_unstale":"2024-05-11T01:14:00.421319+0000","last_undegraded":"2024-05-11T01:14:00.421319+0000","last_fullsized":"2024-05-11T01:14:00.421319+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:34:51.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421726+0000","last_change":"2024-05-11T01:13:59.449973+0000","last_active":"2024-05-11T01:14:00.421726+0000","last_peered":"2024-05-11T01:14:00.421726+0000","last_clean":"2024-05-11T01:14:00.421726+0000","last_became_active":"2024-05-11T01:13:59.449670+0000","last_became_peered":"2024-05-11T01:13:59.449670+0000","last_unstale":"2024-05-11T01:14:00.421726+0000","last_undegraded":"2024-05-11T01:14:00.421726+0000","last_fullsized":"2024-05-11T01:14:00.421726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.422340+0000","last_change":"2024-05-11T01:13:59.450134+0000","last_active":"2024-05-11T01:14:00.422340+0000","last_peered":"2024-05-11T01:14:00.422340+0000","last_clean":"2024-05-11T01:14:00.422340+0000","last_became_active":"2024-05-11T01:13:59.449822+0000","last_became_peered":"2024-05-11T01:13:59.449822+0000","last_unstale":"2024-05-11T01:14:00.422340+0000","last_undegraded":"2024-05-11T01:14:00.422340+0000","last_fullsized":"2024-05-11T01:14:00.422340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421646+0000","last_change":"2024-05-11T01:13:59.444301+0000","last_active":"2024-05-11T01:14:00.421646+0000","last_peered":"2024-05-11T01:14:00.421646+0000","last_clean":"2024-05-11T01:14:00.421646+0000","last_became_active":"2024-05-11T01:13:59.444006+0000","last_became_peered":"2024-05-11T01:13:59.444006+0000","last_unstale":"2024-05-11T01:14:00.421646+0000","last_undegraded":"2024-05-11T01:14:00.421646+0000","last_fullsized":"2024-05-11T01:14:00.421646+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:37:44.925797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425278+0000","last_change":"2024-05-11T01:13:59.428513+0000","last_active":"2024-05-11T01:14:00.425278+0000","last_peered":"2024-05-11T01:14:00.425278+0000","last_clean":"2024-05-11T01:14:00.425278+0000","last_became_active":"2024-05-11T01:13:59.428202+0000","last_became_peered":"2024-05-11T01:13:59.428202+0000","last_unstale":"2024-05-11T01:14:00.425278+0000","last_undegraded":"2024-05-11T01:14:00.425278+0000","last_fullsized":"2024-05-11T01:14:00.425278+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:23.500916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424674+0000","last_change":"2024-05-11T01:13:59.435402+0000","last_active":"2024-05-11T01:14:00.424674+0000","last_peered":"2024-05-11T01:14:00.424674+0000","last_clean":"2024-05-11T01:14:00.424674+0000","last_became_active":"2024-05-11T01:13:59.433324+0000","last_became_peered":"2024-05-11T01:13:59.433324+0000","last_unstale":"2024-05-11T01:14:00.424674+0000","last_undegraded":"2024-05-11T01:14:00.424674+0000","last_fullsized":"2024-05-11T01:14:00.424674+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:36:46.612605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421779+0000","last_change":"2024-05-11T01:13:59.447113+0000","last_active":"2024-05-11T01:14:00.421779+0000","last_peered":"2024-05-11T01:14:00.421779+0000","last_clean":"2024-05-11T01:14:00.421779+0000","last_became_active":"2024-05-11T01:13:59.446349+0000","last_became_peered":"2024-05-11T01:13:59.446349+0000","last_unstale":"2024-05-11T01:14:00.421779+0000","last_undegraded":"2024-05-11T01:14:00.421779+0000","last_fullsized":"2024-05-11T01:14:00.421779+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424048+0000","last_change":"2024-05-11T01:13:59.437811+0000","last_active":"2024-05-11T01:14:00.424048+0000","last_peered":"2024-05-11T01:14:00.424048+0000","last_clean":"2024-05-11T01:14:00.424048+0000","last_became_active":"2024-05-11T01:13:59.427975+0000","last_became_peered":"2024-05-11T01:13:59.427975+0000","last_unstale":"2024-05-11T01:14:00.424048+0000","last_undegraded":"2024-05-11T01:14:00.424048+0000","last_fullsized":"2024-05-11T01:14:00.424048+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:05:27.108182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424221+0000","last_change":"2024-05-11T01:13:59.437261+0000","last_active":"2024-05-11T01:14:00.424221+0000","last_peered":"2024-05-11T01:14:00.424221+0000","last_clean":"2024-05-11T01:14:00.424221+0000","last_became_active":"2024-05-11T01:13:59.436992+0000","last_became_peered":"2024-05-11T01:13:59.436992+0000","last_unstale":"2024-05-11T01:14:00.424221+0000","last_undegraded":"2024-05-11T01:14:00.424221+0000","last_fullsized":"2024-05-11T01:14:00.424221+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:53.116569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424262+0000","last_change":"2024-05-11T01:13:59.445508+0000","last_active":"2024-05-11T01:14:00.424262+0000","last_peered":"2024-05-11T01:14:00.424262+0000","last_clean":"2024-05-11T01:14:00.424262+0000","last_became_active":"2024-05-11T01:13:59.445318+0000","last_became_peered":"2024-05-11T01:13:59.445318+0000","last_unstale":"2024-05-11T01:14:00.424262+0000","last_undegraded":"2024-05-11T01:14:00.424262+0000","last_fullsized":"2024-05-11T01:14:00.424262+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:21.154101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424632+0000","last_change":"2024-05-11T01:13:59.444231+0000","last_active":"2024-05-11T01:14:00.424632+0000","last_peered":"2024-05-11T01:14:00.424632+0000","last_clean":"2024-05-11T01:14:00.424632+0000","last_became_active":"2024-05-11T01:13:59.443762+0000","last_became_peered":"2024-05-11T01:13:59.443762+0000","last_unstale":"2024-05-11T01:14:00.424632+0000","last_undegraded":"2024-05-11T01:14:00.424632+0000","last_fullsized":"2024-05-11T01:14:00.424632+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:25:17.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424359+0000","last_change":"2024-05-11T01:13:59.437781+0000","last_active":"2024-05-11T01:14:00.424359+0000","last_peered":"2024-05-11T01:14:00.424359+0000","last_clean":"2024-05-11T01:14:00.424359+0000","last_became_active":"2024-05-11T01:13:59.437444+0000","last_became_peered":"2024-05-11T01:13:59.437444+0000","last_unstale":"2024-05-11T01:14:00.424359+0000","last_undegraded":"2024-05-11T01:14:00.424359+0000","last_fullsized":"2024-05-11T01:14:00.424359+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:31:12.326102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421934+0000","last_change":"2024-05-11T01:13:59.447343+0000","last_active":"2024-05-11T01:14:00.421934+0000","last_peered":"2024-05-11T01:14:00.421934+0000","last_clean":"2024-05-11T01:14:00.421934+0000","last_became_active":"2024-05-11T01:13:59.446779+0000","last_became_peered":"2024-05-11T01:13:59.446779+0000","last_unstale":"2024-05-11T01:14:00.421934+0000","last_undegraded":"2024-05-11T01:14:00.421934+0000","last_fullsized":"2024-05-11T01:14:00.421934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424031+0000","last_change":"2024-05-11T01:13:59.441132+0000","last_active":"2024-05-11T01:14:00.424031+0000","last_peered":"2024-05-11T01:14:00.424031+0000","last_clean":"2024-05-11T01:14:00.424031+0000","last_became_active":"2024-05-11T01:13:59.440852+0000","last_became_peered":"2024-05-11T01:13:59.440852+0000","last_unstale":"2024-05-11T01:14:00.424031+0000","last_undegraded":"2024-05-11T01:14:00.424031+0000","last_fullsized":"2024-05-11T01:14:00.424031+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:18:58.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424381+0000","last_change":"2024-05-11T01:13:59.436547+0000","last_active":"2024-05-11T01:14:00.424381+0000","last_peered":"2024-05-11T01:14:00.424381+0000","last_clean":"2024-05-11T01:14:00.424381+0000","last_became_active":"2024-05-11T01:13:59.436260+0000","last_became_peered":"2024-05-11T01:13:59.436260+0000","last_unstale":"2024-05-11T01:14:00.424381+0000","last_undegraded":"2024-05-11T01:14:00.424381+0000","last_fullsized":"2024-05-11T01:14:00.424381+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:27.176219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424820+0000","last_change":"2024-05-11T01:13:59.425177+0000","last_active":"2024-05-11T01:14:00.424820+0000","last_peered":"2024-05-11T01:14:00.424820+0000","last_clean":"2024-05-11T01:14:00.424820+0000","last_became_active":"2024-05-11T01:13:59.424328+0000","last_became_peered":"2024-05-11T01:13:59.424328+0000","last_unstale":"2024-05-11T01:14:00.424820+0000","last_undegraded":"2024-05-11T01:14:00.424820+0000","last_fullsized":"2024-05-11T01:14:00.424820+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:29.739616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423903+0000","last_change":"2024-05-11T01:13:59.435398+0000","last_active":"2024-05-11T01:14:00.423903+0000","last_peered":"2024-05-11T01:14:00.423903+0000","last_clean":"2024-05-11T01:14:00.423903+0000","last_became_active":"2024-05-11T01:13:59.435148+0000","last_became_peered":"2024-05-11T01:13:59.435148+0000","last_unstale":"2024-05-11T01:14:00.423903+0000","last_undegraded":"2024-05-11T01:14:00.423903+0000","last_fullsized":"2024-05-11T01:14:00.423903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:14:43.435242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424198+0000","last_change":"2024-05-11T01:13:59.438524+0000","last_active":"2024-05-11T01:14:00.424198+0000","last_peered":"2024-05-11T01:14:00.424198+0000","last_clean":"2024-05-11T01:14:00.424198+0000","last_became_active":"2024-05-11T01:13:59.438262+0000","last_became_peered":"2024-05-11T01:13:59.438262+0000","last_unstale":"2024-05-11T01:14:00.424198+0000","last_undegraded":"2024-05-11T01:14:00.424198+0000","last_fullsized":"2024-05-11T01:14:00.424198+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:44:46.930438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424240+0000","last_change":"2024-05-11T01:13:59.432090+0000","last_active":"2024-05-11T01:14:00.424240+0000","last_peered":"2024-05-11T01:14:00.424240+0000","last_clean":"2024-05-11T01:14:00.424240+0000","last_became_active":"2024-05-11T01:13:59.431691+0000","last_became_peered":"2024-05-11T01:13:59.431691+0000","last_unstale":"2024-05-11T01:14:00.424240+0000","last_undegraded":"2024-05-11T01:14:00.424240+0000","last_fullsized":"2024-05-11T01:14:00.424240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:11:10.857350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425306+0000","last_change":"2024-05-11T01:13:59.440442+0000","last_active":"2024-05-11T01:14:00.425306+0000","last_peered":"2024-05-11T01:14:00.425306+0000","last_clean":"2024-05-11T01:14:00.425306+0000","last_became_active":"2024-05-11T01:13:59.440107+0000","last_became_peered":"2024-05-11T01:13:59.440107+0000","last_unstale":"2024-05-11T01:14:00.425306+0000","last_undegraded":"2024-05-11T01:14:00.425306+0000","last_fullsized":"2024-05-11T01:14:00.425306+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:58:40.948977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424212+0000","last_change":"2024-05-11T01:13:58.440158+0000","last_active":"2024-05-11T01:14:00.424212+0000","last_peered":"2024-05-11T01:14:00.424212+0000","last_clean":"2024-05-11T01:14:00.424212+0000","last_became_active":"2024-05-11T01:13:58.439841+0000","last_became_peered":"2024-05-11T01:13:58.439841+0000","last_unstale":"2024-05-11T01:14:00.424212+0000","last_undegraded":"2024-05-11T01:14:00.424212+0000","last_fullsized":"2024-05-11T01:14:00.424212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424500+0000","last_change":"2024-05-11T01:13:58.433974+0000","last_active":"2024-05-11T01:14:00.424500+0000","last_peered":"2024-05-11T01:14:00.424500+0000","last_clean":"2024-05-11T01:14:00.424500+0000","last_became_active":"2024-05-11T01:13:58.433649+0000","last_became_peered":"2024-05-11T01:13:58.433649+0000","last_unstale":"2024-05-11T01:14:00.424500+0000","last_undegraded":"2024-05-11T01:14:00.424500+0000","last_fullsized":"2024-05-11T01:14:00.424500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:59.322532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424977+0000","last_change":"2024-05-11T01:13:59.439364+0000","last_active":"2024-05-11T01:14:00.424977+0000","last_peered":"2024-05-11T01:14:00.424977+0000","last_clean":"2024-05-11T01:14:00.424977+0000","last_became_active":"2024-05-11T01:13:59.439073+0000","last_became_peered":"2024-05-11T01:13:59.439073+0000","last_unstale":"2024-05-11T01:14:00.424977+0000","last_undegraded":"2024-05-11T01:14:00.424977+0000","last_fullsized":"2024-05-11T01:14:00.424977+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:48.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425486+0000","last_change":"2024-05-11T01:13:58.435449+0000","last_active":"2024-05-11T01:14:00.425486+0000","last_peered":"2024-05-11T01:14:00.425486+0000","last_clean":"2024-05-11T01:14:00.425486+0000","last_became_active":"2024-05-11T01:13:58.434345+0000","last_became_peered":"2024-05-11T01:13:58.434345+0000","last_unstale":"2024-05-11T01:14:00.425486+0000","last_undegraded":"2024-05-11T01:14:00.425486+0000","last_fullsized":"2024-05-11T01:14:00.425486+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:08:58.718268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425493+0000","last_change":"2024-05-11T01:13:59.440315+0000","last_active":"2024-05-11T01:14:00.425493+0000","last_peered":"2024-05-11T01:14:00.425493+0000","last_clean":"2024-05-11T01:14:00.425493+0000","last_became_active":"2024-05-11T01:13:59.439153+0000","last_became_peered":"2024-05-11T01:13:59.439153+0000","last_unstale":"2024-05-11T01:14:00.425493+0000","last_undegraded":"2024-05-11T01:14:00.425493+0000","last_fullsized":"2024-05-11T01:14:00.425493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:38:04.924508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425916+0000","last_change":"2024-05-11T01:13:58.434357+0000","last_active":"2024-05-11T01:14:00.425916+0000","last_peered":"2024-05-11T01:14:00.425916+0000","last_clean":"2024-05-11T01:14:00.425916+0000","last_became_active":"2024-05-11T01:13:58.433392+0000","last_became_peered":"2024-05-11T01:13:58.433392+0000","last_unstale":"2024-05-11T01:14:00.425916+0000","last_undegraded":"2024-05-11T01:14:00.425916+0000","last_fullsized":"2024-05-11T01:14:00.425916+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:14.276107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424274+0000","last_change":"2024-05-11T01:13:59.435145+0000","last_active":"2024-05-11T01:14:00.424274+0000","last_peered":"2024-05-11T01:14:00.424274+0000","last_clean":"2024-05-11T01:14:00.424274+0000","last_became_active":"2024-05-11T01:13:59.434862+0000","last_became_peered":"2024-05-11T01:13:59.434862+0000","last_unstale":"2024-05-11T01:14:00.424274+0000","last_undegraded":"2024-05-11T01:14:00.424274+0000","last_fullsized":"2024-05-11T01:14:00.424274+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:17.395853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424640+0000","last_change":"2024-05-11T01:13:58.439588+0000","last_active":"2024-05-11T01:14:00.424640+0000","last_peered":"2024-05-11T01:14:00.424640+0000","last_clean":"2024-05-11T01:14:00.424640+0000","last_became_active":"2024-05-11T01:13:58.439133+0000","last_became_peered":"2024-05-11T01:13:58.439133+0000","last_unstale":"2024-05-11T01:14:00.424640+0000","last_undegraded":"2024-05-11T01:14:00.424640+0000","last_fullsized":"2024-05-11T01:14:00.424640+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421983+0000","last_change":"2024-05-11T01:13:59.448882+0000","last_active":"2024-05-11T01:14:00.421983+0000","last_peered":"2024-05-11T01:14:00.421983+0000","last_clean":"2024-05-11T01:14:00.421983+0000","last_became_active":"2024-05-11T01:13:59.448208+0000","last_became_peered":"2024-05-11T01:13:59.448208+0000","last_unstale":"2024-05-11T01:14:00.421983+0000","last_undegraded":"2024-05-11T01:14:00.421983+0000","last_fullsized":"2024-05-11T01:14:00.421983+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424024+0000","last_change":"2024-05-11T01:13:58.428838+0000","last_active":"2024-05-11T01:14:00.424024+0000","last_peered":"2024-05-11T01:14:00.424024+0000","last_clean":"2024-05-11T01:14:00.424024+0000","last_became_active":"2024-05-11T01:13:58.428556+0000","last_became_peered":"2024-05-11T01:13:58.428556+0000","last_unstale":"2024-05-11T01:14:00.424024+0000","last_undegraded":"2024-05-11T01:14:00.424024+0000","last_fullsized":"2024-05-11T01:14:00.424024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:01:35.971417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421389+0000","last_change":"2024-05-11T01:13:59.447010+0000","last_active":"2024-05-11T01:14:00.421389+0000","last_peered":"2024-05-11T01:14:00.421389+0000","last_clean":"2024-05-11T01:14:00.421389+0000","last_became_active":"2024-05-11T01:13:59.446713+0000","last_became_peered":"2024-05-11T01:13:59.446713+0000","last_unstale":"2024-05-11T01:14:00.421389+0000","last_undegraded":"2024-05-11T01:14:00.421389+0000","last_fullsized":"2024-05-11T01:14:00.421389+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:03:04.844818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424193+0000","last_change":"2024-05-11T01:13:58.428031+0000","last_active":"2024-05-11T01:14:00.424193+0000","last_peered":"2024-05-11T01:14:00.424193+0000","last_clean":"2024-05-11T01:14:00.424193+0000","last_became_active":"2024-05-11T01:13:58.427742+0000","last_became_peered":"2024-05-11T01:13:58.427742+0000","last_unstale":"2024-05-11T01:14:00.424193+0000","last_undegraded":"2024-05-11T01:14:00.424193+0000","last_fullsized":"2024-05-11T01:14:00.424193+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421412+0000","last_change":"2024-05-11T01:13:58.436246+0000","last_active":"2024-05-11T01:14:00.421412+0000","last_peered":"2024-05-11T01:14:00.421412+0000","last_clean":"2024-05-11T01:14:00.421412+0000","last_became_active":"2024-05-11T01:13:58.435967+0000","last_became_peered":"2024-05-11T01:13:58.435967+0000","last_unstale":"2024-05-11T01:14:00.421412+0000","last_undegraded":"2024-05-11T01:14:00.421412+0000","last_fullsized":"2024-05-11T01:14:00.421412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:22.242555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424432+0000","last_change":"2024-05-11T01:13:59.445506+0000","last_active":"2024-05-11T01:14:00.424432+0000","last_peered":"2024-05-11T01:14:00.424432+0000","last_clean":"2024-05-11T01:14:00.424432+0000","last_became_active":"2024-05-11T01:13:59.445232+0000","last_became_peered":"2024-05-11T01:13:59.445232+0000","last_unstale":"2024-05-11T01:14:00.424432+0000","last_undegraded":"2024-05-11T01:14:00.424432+0000","last_fullsized":"2024-05-11T01:14:00.424432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:18:27.371597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424424+0000","last_change":"2024-05-11T01:13:58.426187+0000","last_active":"2024-05-11T01:14:00.424424+0000","last_peered":"2024-05-11T01:14:00.424424+0000","last_clean":"2024-05-11T01:14:00.424424+0000","last_became_active":"2024-05-11T01:13:58.425913+0000","last_became_peered":"2024-05-11T01:13:58.425913+0000","last_unstale":"2024-05-11T01:14:00.424424+0000","last_undegraded":"2024-05-11T01:14:00.424424+0000","last_fullsized":"2024-05-11T01:14:00.424424+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426499+0000","last_change":"2024-05-11T01:13:59.431401+0000","last_active":"2024-05-11T01:14:00.426499+0000","last_peered":"2024-05-11T01:14:00.426499+0000","last_clean":"2024-05-11T01:14:00.426499+0000","last_became_active":"2024-05-11T01:13:59.431126+0000","last_became_peered":"2024-05-11T01:13:59.431126+0000","last_unstale":"2024-05-11T01:14:00.426499+0000","last_undegraded":"2024-05-11T01:14:00.426499+0000","last_fullsized":"2024-05-11T01:14:00.426499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:42:47.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.422678+0000","last_change":"2024-05-11T01:13:58.431417+0000","last_active":"2024-05-11T01:14:00.422678+0000","last_peered":"2024-05-11T01:14:00.422678+0000","last_clean":"2024-05-11T01:14:00.422678+0000","last_became_active":"2024-05-11T01:13:58.431132+0000","last_became_peered":"2024-05-11T01:13:58.431132+0000","last_unstale":"2024-05-11T01:14:00.422678+0000","last_undegraded":"2024-05-11T01:14:00.422678+0000","last_fullsized":"2024-05-11T01:14:00.422678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:23:16.977274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424523+0000","last_change":"2024-05-11T01:13:59.436396+0000","last_active":"2024-05-11T01:14:00.424523+0000","last_peered":"2024-05-11T01:14:00.424523+0000","last_clean":"2024-05-11T01:14:00.424523+0000","last_became_active":"2024-05-11T01:13:59.436043+0000","last_became_peered":"2024-05-11T01:13:59.436043+0000","last_unstale":"2024-05-11T01:14:00.424523+0000","last_undegraded":"2024-05-11T01:14:00.424523+0000","last_fullsized":"2024-05-11T01:14:00.424523+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426232+0000","last_change":"2024-05-11T01:13:58.435566+0000","last_active":"2024-05-11T01:14:00.426232+0000","last_peered":"2024-05-11T01:14:00.426232+0000","last_clean":"2024-05-11T01:14:00.426232+0000","last_became_active":"2024-05-11T01:13:58.435280+0000","last_became_peered":"2024-05-11T01:13:58.435280+0000","last_unstale":"2024-05-11T01:14:00.426232+0000","last_undegraded":"2024-05-11T01:14:00.426232+0000","last_fullsized":"2024-05-11T01:14:00.426232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:47.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425975+0000","last_change":"2024-05-11T01:13:59.432307+0000","last_active":"2024-05-11T01:14:00.425975+0000","last_peered":"2024-05-11T01:14:00.425975+0000","last_clean":"2024-05-11T01:14:00.425975+0000","last_became_active":"2024-05-11T01:13:59.431913+0000","last_became_peered":"2024-05-11T01:13:59.431913+0000","last_unstale":"2024-05-11T01:14:00.425975+0000","last_undegraded":"2024-05-11T01:14:00.425975+0000","last_fullsized":"2024-05-11T01:14:00.425975+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"60'6","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424733+0000","last_change":"2024-05-11T01:13:58.425157+0000","last_active":"2024-05-11T01:14:00.424733+0000","last_peered":"2024-05-11T01:14:00.424733+0000","last_clean":"2024-05-11T01:14:00.424733+0000","last_became_active":"2024-05-11T01:13:58.424894+0000","last_became_peered":"2024-05-11T01:13:58.424894+0000","last_unstale":"2024-05-11T01:14:00.424733+0000","last_undegraded":"2024-05-11T01:14:00.424733+0000","last_fullsized":"2024-05-11T01:14:00.424733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:48:11.424880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424741+0000","last_change":"2024-05-11T01:13:58.439044+0000","last_active":"2024-05-11T01:14:00.424741+0000","last_peered":"2024-05-11T01:14:00.424741+0000","last_clean":"2024-05-11T01:14:00.424741+0000","last_became_active":"2024-05-11T01:13:58.438603+0000","last_became_peered":"2024-05-11T01:13:58.438603+0000","last_unstale":"2024-05-11T01:14:00.424741+0000","last_undegraded":"2024-05-11T01:14:00.424741+0000","last_fullsized":"2024-05-11T01:14:00.424741+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421937+0000","last_change":"2024-05-11T01:13:59.449114+0000","last_active":"2024-05-11T01:14:00.421937+0000","last_peered":"2024-05-11T01:14:00.421937+0000","last_clean":"2024-05-11T01:14:00.421937+0000","last_became_active":"2024-05-11T01:13:59.448532+0000","last_became_peered":"2024-05-11T01:13:59.448532+0000","last_unstale":"2024-05-11T01:14:00.421937+0000","last_undegraded":"2024-05-11T01:14:00.421937+0000","last_fullsized":"2024-05-11T01:14:00.421937+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425738+0000","last_change":"2024-05-11T01:13:58.434118+0000","last_active":"2024-05-11T01:14:00.425738+0000","last_peered":"2024-05-11T01:14:00.425738+0000","last_clean":"2024-05-11T01:14:00.425738+0000","last_became_active":"2024-05-11T01:13:58.433818+0000","last_became_peered":"2024-05-11T01:13:58.433818+0000","last_unstale":"2024-05-11T01:14:00.425738+0000","last_undegraded":"2024-05-11T01:14:00.425738+0000","last_fullsized":"2024-05-11T01:14:00.425738+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:49.867304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423798+0000","last_change":"2024-05-11T01:13:59.435013+0000","last_active":"2024-05-11T01:14:00.423798+0000","last_peered":"2024-05-11T01:14:00.423798+0000","last_clean":"2024-05-11T01:14:00.423798+0000","last_became_active":"2024-05-11T01:13:59.434726+0000","last_became_peered":"2024-05-11T01:13:59.434726+0000","last_unstale":"2024-05-11T01:14:00.423798+0000","last_undegraded":"2024-05-11T01:14:00.423798+0000","last_fullsized":"2024-05-11T01:14:00.423798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:03:22.763694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424136+0000","last_change":"2024-05-11T01:13:58.438894+0000","last_active":"2024-05-11T01:14:00.424136+0000","last_peered":"2024-05-11T01:14:00.424136+0000","last_clean":"2024-05-11T01:14:00.424136+0000","last_became_active":"2024-05-11T01:13:58.438528+0000","last_became_peered":"2024-05-11T01:13:58.438528+0000","last_unstale":"2024-05-11T01:14:00.424136+0000","last_undegraded":"2024-05-11T01:14:00.424136+0000","last_fullsized":"2024-05-11T01:14:00.424136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426033+0000","last_change":"2024-05-11T01:13:59.437777+0000","last_active":"2024-05-11T01:14:00.426033+0000","last_peered":"2024-05-11T01:14:00.426033+0000","last_clean":"2024-05-11T01:14:00.426033+0000","last_became_active":"2024-05-11T01:13:59.437176+0000","last_became_peered":"2024-05-11T01:13:59.437176+0000","last_unstale":"2024-05-11T01:14:00.426033+0000","last_undegraded":"2024-05-11T01:14:00.426033+0000","last_fullsized":"2024-05-11T01:14:00.426033+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:07:21.319542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421494+0000","last_change":"2024-05-11T01:13:58.431218+0000","last_active":"2024-05-11T01:14:00.421494+0000","last_peered":"2024-05-11T01:14:00.421494+0000","last_clean":"2024-05-11T01:14:00.421494+0000","last_became_active":"2024-05-11T01:13:58.430952+0000","last_became_peered":"2024-05-11T01:13:58.430952+0000","last_unstale":"2024-05-11T01:14:00.421494+0000","last_undegraded":"2024-05-11T01:14:00.421494+0000","last_fullsized":"2024-05-11T01:14:00.421494+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:03:39.335329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424175+0000","last_change":"2024-05-11T01:13:59.427029+0000","last_active":"2024-05-11T01:14:00.424175+0000","last_peered":"2024-05-11T01:14:00.424175+0000","last_clean":"2024-05-11T01:14:00.424175+0000","last_became_active":"2024-05-11T01:13:59.426727+0000","last_became_peered":"2024-05-11T01:13:59.426727+0000","last_unstale":"2024-05-11T01:14:00.424175+0000","last_undegraded":"2024-05-11T01:14:00.424175+0000","last_fullsized":"2024-05-11T01:14:00.424175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:52.626350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424360+0000","last_change":"2024-05-11T01:13:58.434507+0000","last_active":"2024-05-11T01:14:00.424360+0000","last_peered":"2024-05-11T01:14:00.424360+0000","last_clean":"2024-05-11T01:14:00.424360+0000","last_became_active":"2024-05-11T01:13:58.434215+0000","last_became_peered":"2024-05-11T01:13:58.434215+0000","last_unstale":"2024-05-11T01:14:00.424360+0000","last_undegraded":"2024-05-11T01:14:00.424360+0000","last_fullsized":"2024-05-11T01:14:00.424360+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:35.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421887+0000","last_change":"2024-05-11T01:13:59.444388+0000","last_active":"2024-05-11T01:14:00.421887+0000","last_peered":"2024-05-11T01:14:00.421887+0000","last_clean":"2024-05-11T01:14:00.421887+0000","last_became_active":"2024-05-11T01:13:59.443981+0000","last_became_peered":"2024-05-11T01:13:59.443981+0000","last_unstale":"2024-05-11T01:14:00.421887+0000","last_undegraded":"2024-05-11T01:14:00.421887+0000","last_fullsized":"2024-05-11T01:14:00.421887+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425048+0000","last_change":"2024-05-11T01:13:58.429801+0000","last_active":"2024-05-11T01:14:00.425048+0000","last_peered":"2024-05-11T01:14:00.425048+0000","last_clean":"2024-05-11T01:14:00.425048+0000","last_became_active":"2024-05-11T01:13:58.429538+0000","last_became_peered":"2024-05-11T01:13:58.429538+0000","last_unstale":"2024-05-11T01:14:00.425048+0000","last_undegraded":"2024-05-11T01:14:00.425048+0000","last_fullsized":"2024-05-11T01:14:00.425048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:27.314762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424362+0000","last_change":"2024-05-11T01:13:58.426925+0000","last_active":"2024-05-11T01:14:00.424362+0000","last_peered":"2024-05-11T01:14:00.424362+0000","last_clean":"2024-05-11T01:14:00.424362+0000","last_became_active":"2024-05-11T01:13:58.426586+0000","last_became_peered":"2024-05-11T01:13:58.426586+0000","last_unstale":"2024-05-11T01:14:00.424362+0000","last_undegraded":"2024-05-11T01:14:00.424362+0000","last_fullsized":"2024-05-11T01:14:00.424362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424523+0000","last_change":"2024-05-11T01:13:58.439680+0000","last_active":"2024-05-11T01:14:00.424523+0000","last_peered":"2024-05-11T01:14:00.424523+0000","last_clean":"2024-05-11T01:14:00.424523+0000","last_became_active":"2024-05-11T01:13:58.439404+0000","last_became_peered":"2024-05-11T01:13:58.439404+0000","last_unstale":"2024-05-11T01:14:00.424523+0000","last_undegraded":"2024-05-11T01:14:00.424523+0000","last_fullsized":"2024-05-11T01:14:00.424523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:41:26.236385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424081+0000","last_change":"2024-05-11T01:13:59.435753+0000","last_active":"2024-05-11T01:14:00.424081+0000","last_peered":"2024-05-11T01:14:00.424081+0000","last_clean":"2024-05-11T01:14:00.424081+0000","last_became_active":"2024-05-11T01:13:59.435253+0000","last_became_peered":"2024-05-11T01:13:59.435253+0000","last_unstale":"2024-05-11T01:14:00.424081+0000","last_undegraded":"2024-05-11T01:14:00.424081+0000","last_fullsized":"2024-05-11T01:14:00.424081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:06:58.758258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424848+0000","last_change":"2024-05-11T01:13:58.437862+0000","last_active":"2024-05-11T01:14:00.424848+0000","last_peered":"2024-05-11T01:14:00.424848+0000","last_clean":"2024-05-11T01:14:00.424848+0000","last_became_active":"2024-05-11T01:13:58.437597+0000","last_became_peered":"2024-05-11T01:13:58.437597+0000","last_unstale":"2024-05-11T01:14:00.424848+0000","last_undegraded":"2024-05-11T01:14:00.424848+0000","last_fullsized":"2024-05-11T01:14:00.424848+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:19:23.818943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425229+0000","last_change":"2024-05-11T01:13:59.431673+0000","last_active":"2024-05-11T01:14:00.425229+0000","last_peered":"2024-05-11T01:14:00.425229+0000","last_clean":"2024-05-11T01:14:00.425229+0000","last_became_active":"2024-05-11T01:13:59.431386+0000","last_became_peered":"2024-05-11T01:13:59.431386+0000","last_unstale":"2024-05-11T01:14:00.425229+0000","last_undegraded":"2024-05-11T01:14:00.425229+0000","last_fullsized":"2024-05-11T01:14:00.425229+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:27.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424097+0000","last_change":"2024-05-11T01:13:58.431189+0000","last_active":"2024-05-11T01:14:00.424097+0000","last_peered":"2024-05-11T01:14:00.424097+0000","last_clean":"2024-05-11T01:14:00.424097+0000","last_became_active":"2024-05-11T01:13:58.430890+0000","last_became_peered":"2024-05-11T01:13:58.430890+0000","last_unstale":"2024-05-11T01:14:00.424097+0000","last_undegraded":"2024-05-11T01:14:00.424097+0000","last_fullsized":"2024-05-11T01:14:00.424097+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424319+0000","last_change":"2024-05-11T01:13:59.429997+0000","last_active":"2024-05-11T01:14:00.424319+0000","last_peered":"2024-05-11T01:14:00.424319+0000","last_clean":"2024-05-11T01:14:00.424319+0000","last_became_active":"2024-05-11T01:13:59.429734+0000","last_became_peered":"2024-05-11T01:13:59.429734+0000","last_unstale":"2024-05-11T01:14:00.424319+0000","last_undegraded":"2024-05-11T01:14:00.424319+0000","last_fullsized":"2024-05-11T01:14:00.424319+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:47.445191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421543+0000","last_change":"2024-05-11T01:13:58.431136+0000","last_active":"2024-05-11T01:14:00.421543+0000","last_peered":"2024-05-11T01:14:00.421543+0000","last_clean":"2024-05-11T01:14:00.421543+0000","last_became_active":"2024-05-11T01:13:58.430846+0000","last_became_peered":"2024-05-11T01:13:58.430846+0000","last_unstale":"2024-05-11T01:14:00.421543+0000","last_undegraded":"2024-05-11T01:14:00.421543+0000","last_fullsized":"2024-05-11T01:14:00.421543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:38.439492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421440+0000","last_change":"2024-05-11T01:13:59.450616+0000","last_active":"2024-05-11T01:14:00.421440+0000","last_peered":"2024-05-11T01:14:00.421440+0000","last_clean":"2024-05-11T01:14:00.421440+0000","last_became_active":"2024-05-11T01:13:59.450300+0000","last_became_peered":"2024-05-11T01:13:59.450300+0000","last_unstale":"2024-05-11T01:14:00.421440+0000","last_undegraded":"2024-05-11T01:14:00.421440+0000","last_fullsized":"2024-05-11T01:14:00.421440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425506+0000","last_change":"2024-05-11T01:13:58.436187+0000","last_active":"2024-05-11T01:14:00.425506+0000","last_peered":"2024-05-11T01:14:00.425506+0000","last_clean":"2024-05-11T01:14:00.425506+0000","last_became_active":"2024-05-11T01:13:58.434702+0000","last_became_peered":"2024-05-11T01:13:58.434702+0000","last_unstale":"2024-05-11T01:14:00.425506+0000","last_undegraded":"2024-05-11T01:14:00.425506+0000","last_fullsized":"2024-05-11T01:14:00.425506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:28.737362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T01:13:58.410869+0000","last_change":"2024-05-11T01:13:58.410869+0000","last_active":"2024-05-11T01:13:58.410869+0000","last_peered":"2024-05-11T01:13:58.410869+0000","last_clean":"2024-05-11T01:13:58.410869+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T01:13:58.410869+0000","last_undegraded":"2024-05-11T01:13:58.410869+0000","last_fullsized":"2024-05-11T01:13:58.410869+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"61'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.428543+0000","last_change":"2024-05-11T01:13:58.436921+0000","last_active":"2024-05-11T01:14:00.428543+0000","last_peered":"2024-05-11T01:14:00.428543+0000","last_clean":"2024-05-11T01:14:00.428543+0000","last_became_active":"2024-05-11T01:13:58.436650+0000","last_became_peered":"2024-05-11T01:13:58.436650+0000","last_unstale":"2024-05-11T01:14:00.428543+0000","last_undegraded":"2024-05-11T01:14:00.428543+0000","last_fullsized":"2024-05-11T01:14:00.428543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:16:06.268571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421915+0000","last_change":"2024-05-11T01:13:59.442952+0000","last_active":"2024-05-11T01:14:00.421915+0000","last_peered":"2024-05-11T01:14:00.421915+0000","last_clean":"2024-05-11T01:14:00.421915+0000","last_became_active":"2024-05-11T01:13:59.442413+0000","last_became_peered":"2024-05-11T01:13:59.442413+0000","last_unstale":"2024-05-11T01:14:00.421915+0000","last_undegraded":"2024-05-11T01:14:00.421915+0000","last_fullsized":"2024-05-11T01:14:00.421915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:32:54.808774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426328+0000","last_change":"2024-05-11T01:13:58.436342+0000","last_active":"2024-05-11T01:14:00.426328+0000","last_peered":"2024-05-11T01:14:00.426328+0000","last_clean":"2024-05-11T01:14:00.426328+0000","last_became_active":"2024-05-11T01:13:58.436037+0000","last_became_peered":"2024-05-11T01:13:58.436037+0000","last_unstale":"2024-05-11T01:14:00.426328+0000","last_undegraded":"2024-05-11T01:14:00.426328+0000","last_fullsized":"2024-05-11T01:14:00.426328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:22.498662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426553+0000","last_change":"2024-05-11T01:13:59.434239+0000","last_active":"2024-05-11T01:14:00.426553+0000","last_peered":"2024-05-11T01:14:00.426553+0000","last_clean":"2024-05-11T01:14:00.426553+0000","last_became_active":"2024-05-11T01:13:59.433946+0000","last_became_peered":"2024-05-11T01:13:59.433946+0000","last_unstale":"2024-05-11T01:14:00.426553+0000","last_undegraded":"2024-05-11T01:14:00.426553+0000","last_fullsized":"2024-05-11T01:14:00.426553+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:07:36.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425174+0000","last_change":"2024-05-11T01:13:58.432458+0000","last_active":"2024-05-11T01:14:00.425174+0000","last_peered":"2024-05-11T01:14:00.425174+0000","last_clean":"2024-05-11T01:14:00.425174+0000","last_became_active":"2024-05-11T01:13:58.432149+0000","last_became_peered":"2024-05-11T01:13:58.432149+0000","last_unstale":"2024-05-11T01:14:00.425174+0000","last_undegraded":"2024-05-11T01:14:00.425174+0000","last_fullsized":"2024-05-11T01:14:00.425174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424224+0000","last_change":"2024-05-11T01:13:59.432345+0000","last_active":"2024-05-11T01:14:00.424224+0000","last_peered":"2024-05-11T01:14:00.424224+0000","last_clean":"2024-05-11T01:14:00.424224+0000","last_became_active":"2024-05-11T01:13:59.432058+0000","last_became_peered":"2024-05-11T01:13:59.432058+0000","last_unstale":"2024-05-11T01:14:00.424224+0000","last_undegraded":"2024-05-11T01:14:00.424224+0000","last_fullsized":"2024-05-11T01:14:00.424224+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:38:32.025100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425955+0000","last_change":"2024-05-11T01:13:58.433877+0000","last_active":"2024-05-11T01:14:00.425955+0000","last_peered":"2024-05-11T01:14:00.425955+0000","last_clean":"2024-05-11T01:14:00.425955+0000","last_became_active":"2024-05-11T01:13:58.433584+0000","last_became_peered":"2024-05-11T01:13:58.433584+0000","last_unstale":"2024-05-11T01:14:00.425955+0000","last_undegraded":"2024-05-11T01:14:00.425955+0000","last_fullsized":"2024-05-11T01:14:00.425955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:20.805314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424533+0000","last_change":"2024-05-11T01:13:59.434011+0000","last_active":"2024-05-11T01:14:00.424533+0000","last_peered":"2024-05-11T01:14:00.424533+0000","last_clean":"2024-05-11T01:14:00.424533+0000","last_became_active":"2024-05-11T01:13:59.432337+0000","last_became_peered":"2024-05-11T01:13:59.432337+0000","last_unstale":"2024-05-11T01:14:00.424533+0000","last_undegraded":"2024-05-11T01:14:00.424533+0000","last_fullsized":"2024-05-11T01:14:00.424533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424127+0000","last_change":"2024-05-11T01:13:58.435211+0000","last_active":"2024-05-11T01:14:00.424127+0000","last_peered":"2024-05-11T01:14:00.424127+0000","last_clean":"2024-05-11T01:14:00.424127+0000","last_became_active":"2024-05-11T01:13:58.434956+0000","last_became_peered":"2024-05-11T01:13:58.434956+0000","last_unstale":"2024-05-11T01:14:00.424127+0000","last_undegraded":"2024-05-11T01:14:00.424127+0000","last_fullsized":"2024-05-11T01:14:00.424127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424617+0000","last_change":"2024-05-11T01:13:59.432319+0000","last_active":"2024-05-11T01:14:00.424617+0000","last_peered":"2024-05-11T01:14:00.424617+0000","last_clean":"2024-05-11T01:14:00.424617+0000","last_became_active":"2024-05-11T01:13:59.432012+0000","last_became_peered":"2024-05-11T01:13:59.432012+0000","last_unstale":"2024-05-11T01:14:00.424617+0000","last_undegraded":"2024-05-11T01:14:00.424617+0000","last_fullsized":"2024-05-11T01:14:00.424617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:21.807568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424991+0000","last_change":"2024-05-11T01:13:58.433752+0000","last_active":"2024-05-11T01:14:00.424991+0000","last_peered":"2024-05-11T01:14:00.424991+0000","last_clean":"2024-05-11T01:14:00.424991+0000","last_became_active":"2024-05-11T01:13:58.433411+0000","last_became_peered":"2024-05-11T01:13:58.433411+0000","last_unstale":"2024-05-11T01:14:00.424991+0000","last_undegraded":"2024-05-11T01:14:00.424991+0000","last_fullsized":"2024-05-11T01:14:00.424991+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423892+0000","last_change":"2024-05-11T01:13:59.446971+0000","last_active":"2024-05-11T01:14:00.423892+0000","last_peered":"2024-05-11T01:14:00.423892+0000","last_clean":"2024-05-11T01:14:00.423892+0000","last_became_active":"2024-05-11T01:13:59.446670+0000","last_became_peered":"2024-05-11T01:13:59.446670+0000","last_unstale":"2024-05-11T01:14:00.423892+0000","last_undegraded":"2024-05-11T01:14:00.423892+0000","last_fullsized":"2024-05-11T01:14:00.423892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:45:46.843918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421509+0000","last_change":"2024-05-11T01:13:58.435876+0000","last_active":"2024-05-11T01:14:00.421509+0000","last_peered":"2024-05-11T01:14:00.421509+0000","last_clean":"2024-05-11T01:14:00.421509+0000","last_became_active":"2024-05-11T01:13:58.434062+0000","last_became_peered":"2024-05-11T01:13:58.434062+0000","last_unstale":"2024-05-11T01:14:00.421509+0000","last_undegraded":"2024-05-11T01:14:00.421509+0000","last_fullsized":"2024-05-11T01:14:00.421509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:12:26.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.422491+0000","last_change":"2024-05-11T01:13:59.445426+0000","last_active":"2024-05-11T01:14:00.422491+0000","last_peered":"2024-05-11T01:14:00.422491+0000","last_clean":"2024-05-11T01:14:00.422491+0000","last_became_active":"2024-05-11T01:13:59.445146+0000","last_became_peered":"2024-05-11T01:13:59.445146+0000","last_unstale":"2024-05-11T01:14:00.422491+0000","last_undegraded":"2024-05-11T01:14:00.422491+0000","last_fullsized":"2024-05-11T01:14:00.422491+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:57:36.879340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424737+0000","last_change":"2024-05-11T01:13:58.434600+0000","last_active":"2024-05-11T01:14:00.424737+0000","last_peered":"2024-05-11T01:14:00.424737+0000","last_clean":"2024-05-11T01:14:00.424737+0000","last_became_active":"2024-05-11T01:13:58.434320+0000","last_became_peered":"2024-05-11T01:13:58.434320+0000","last_unstale":"2024-05-11T01:14:00.424737+0000","last_undegraded":"2024-05-11T01:14:00.424737+0000","last_fullsized":"2024-05-11T01:14:00.424737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"51'86","reported_seq":241,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425450+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:14:00.425450+0000","last_peered":"2024-05-11T01:14:00.425450+0000","last_clean":"2024-05-11T01:14:00.425450+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:14:00.425450+0000","last_undegraded":"2024-05-11T01:14:00.425450+0000","last_fullsized":"2024-05-11T01:14:00.425450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":936289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542304,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31832,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":663552,"data_stored":312240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":729088,"data_stored":345901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575007,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54112,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317728,"statfs":{"total":96636764160,"available":96581353472,"internally_reserved":0,"allocated":737280,"data_stored":347443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44384,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327456,"statfs":{"total":96636764160,"available":96591314944,"internally_reserved":0,"allocated":737280,"data_stored":346469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":347465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58268,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313572,"statfs":{"total":96636764160,"available":96577097728,"internally_reserved":0,"allocated":729088,"data_stored":345901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36640,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335200,"statfs":{"total":96636764160,"available":96599244800,"internally_reserved":0,"allocated":1323008,"data_stored":936233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:14:03.487 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:14:03.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.624+0000 7f6a4979e640 1 Processor -- start 2024-05-11T01:14:03.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.624+0000 7f6a4979e640 1 -- start start 2024-05-11T01:14:03.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.625+0000 7f6a4979e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6a4415f2c0 0x7f6a440cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.625+0000 7f6a4979e640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a440d0210 con 0x7f6a440d2060 2024-05-11T01:14:03.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.625+0000 7f6a4979e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a440d04f0 con 0x7f6a440d3ca0 2024-05-11T01:14:03.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.625+0000 7f6a4979e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f6a440d0e20 con 0x7f6a4415f2c0 2024-05-11T01:14:03.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.625+0000 7f6a43fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f6a4415f2c0 0x7f6a440cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.625+0000 7f6a437fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6a440d2060 0x7f6a440d3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55230/0 (socket says 172.21.15.52:55230) 2024-05-11T01:14:03.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.625+0000 7f6a437fe640 1 -- 172.21.15.52:0/1426699499 learned_addr learned my addr 172.21.15.52:0/1426699499 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:14:03.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.626+0000 7f6a43fff640 1 -- 172.21.15.52:0/1426699499 >> v1:172.21.15.119:6789/0 conn(0x7f6a440d3ca0 legacy=0x7f6a440b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.626+0000 7f6a43fff640 1 -- 172.21.15.52:0/1426699499 >> v1:172.21.15.52:6789/0 conn(0x7f6a440d2060 legacy=0x7f6a440d3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.626+0000 7f6a43fff640 1 -- 172.21.15.52:0/1426699499 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a440d07d0 con 0x7f6a4415f2c0 2024-05-11T01:14:03.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.626+0000 7f6a43fff640 1 --2- 172.21.15.52:0/1426699499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a4415f2c0 0x7f6a440cfb30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6a340097a0 tx=0x7f6a3402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9c567ba5759491 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.626+0000 7f6a42ffd640 1 -- 172.21.15.52:0/1426699499 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34037070 con 0x7f6a4415f2c0 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.626+0000 7f6a42ffd640 1 -- 172.21.15.52:0/1426699499 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a34032640 con 0x7f6a4415f2c0 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.627+0000 7f6a42ffd640 1 -- 172.21.15.52:0/1426699499 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34032a10 con 0x7f6a4415f2c0 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.627+0000 7f6a4979e640 1 -- 172.21.15.52:0/1426699499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a4415f2c0 msgr2=0x7f6a440cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.627+0000 7f6a4979e640 1 --2- 172.21.15.52:0/1426699499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a4415f2c0 0x7f6a440cfb30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6a340097a0 tx=0x7f6a3402a2c0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.627+0000 7f6a4979e640 1 -- 172.21.15.52:0/1426699499 shutdown_connections 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.627+0000 7f6a4979e640 1 --2- 172.21.15.52:0/1426699499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a4415f2c0 0x7f6a440cfb30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.627+0000 7f6a4979e640 1 -- 172.21.15.52:0/1426699499 >> 172.21.15.52:0/1426699499 conn(0x7f6a4409e320 msgr2=0x7f6a440d4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.627+0000 7f6a4979e640 1 -- 172.21.15.52:0/1426699499 shutdown_connections 2024-05-11T01:14:03.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.627+0000 7f6a4979e640 1 -- 172.21.15.52:0/1426699499 wait complete. 2024-05-11T01:14:03.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.628+0000 7f6a4979e640 1 Processor -- start 2024-05-11T01:14:03.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a4979e640 1 -- start start 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a4979e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a440d2060 0x7f6a44161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a4979e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6a440d3ca0 0x7f6a440a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a4979e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a4415f2c0 0x7f6a440b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a4979e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4415f6a0 con 0x7f6a440d2060 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a4979e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a440d24a0 con 0x7f6a4415f2c0 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a4979e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4415f820 con 0x7f6a440d3ca0 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a43fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a440d2060 0x7f6a44161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a43fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a440d2060 0x7f6a44161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49564/0 (socket says 172.21.15.52:49564) 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.629+0000 7f6a43fff640 1 -- 172.21.15.52:0/1754364408 learned_addr learned my addr 172.21.15.52:0/1754364408 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a48f9d640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a4415f2c0 0x7f6a440b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a43fff640 1 -- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6a440d3ca0 msgr2=0x7f6a440a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a43fff640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6a440d3ca0 0x7f6a440a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a43fff640 1 -- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a4415f2c0 msgr2=0x7f6a440b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a43fff640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a4415f2c0 0x7f6a440b7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a43fff640 1 -- 172.21.15.52:0/1754364408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a440a2c10 con 0x7f6a440d2060 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a48f9d640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a4415f2c0 0x7f6a440b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a43fff640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a440d2060 0x7f6a44161d00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6a340098c0 tx=0x7f6a340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a40ff9640 1 -- 172.21.15.52:0/1754364408 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34041030 con 0x7f6a440d2060 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a40ff9640 1 -- 172.21.15.52:0/1754364408 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a34030750 con 0x7f6a440d2060 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a44162b80 con 0x7f6a440d2060 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a40ff9640 1 -- 172.21.15.52:0/1754364408 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34037040 con 0x7f6a440d2060 2024-05-11T01:14:03.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.630+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a441625c0 con 0x7f6a440d2060 2024-05-11T01:14:03.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.631+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a440694f0 con 0x7f6a440d2060 2024-05-11T01:14:03.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.632+0000 7f6a40ff9640 1 -- 172.21.15.52:0/1754364408 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f6a34044070 con 0x7f6a440d2060 2024-05-11T01:14:03.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.632+0000 7f6a40ff9640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6a30076300 0x7f6a300787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.632+0000 7f6a40ff9640 1 -- 172.21.15.52:0/1754364408 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f6a340b8120 con 0x7f6a440d2060 2024-05-11T01:14:03.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.633+0000 7f6a437fe640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6a30076300 0x7f6a300787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.633+0000 7f6a437fe640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6a30076300 0x7f6a300787c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6a38005880 tx=0x7f6a38007a20 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:03.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.637+0000 7f6a40ff9640 1 -- 172.21.15.52:0/1754364408 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6a440a2c10 con 0x7f6a440d2060 2024-05-11T01:14:03.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.825+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a440b9270 con 0x7f6a30076300 2024-05-11T01:14:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.831+0000 7f6a40ff9640 1 -- 172.21.15.52:0/1754364408 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251954 (secure 0 0 0) 0x7f6a440b9270 con 0x7f6a30076300 2024-05-11T01:14:03.833 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:14:03.834 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:14:03.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6a30076300 msgr2=0x7f6a300787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6a30076300 0x7f6a300787c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6a38005880 tx=0x7f6a38007a20 comp rx=0 tx=0).stop 2024-05-11T01:14:03.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a440d2060 msgr2=0x7f6a44161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:03.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a440d2060 0x7f6a44161d00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6a340098c0 tx=0x7f6a340046c0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 shutdown_connections 2024-05-11T01:14:03.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6a30076300 0x7f6a300787c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a4415f2c0 0x7f6a440b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6a440d3ca0 0x7f6a440a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 --2- 172.21.15.52:0/1754364408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a440d2060 0x7f6a44161d00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 >> 172.21.15.52:0/1754364408 conn(0x7f6a4409e320 msgr2=0x7f6a440ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:03.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 shutdown_connections 2024-05-11T01:14:03.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.834+0000 7f6a4979e640 1 -- 172.21.15.52:0/1754364408 wait complete. 2024-05-11T01:14:03.844 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-11T01:14:03.707813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":170,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317424,"kb_used_data":6888,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754657296,"statfs":{"total":773094113280,"available":772769071104,"internally_reserved":0,"allocated":7053312,"data_stored":3952204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":317836849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001172"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426499+0000","last_change":"2024-05-11T01:13:59.434893+0000","last_active":"2024-05-11T01:14:00.426499+0000","last_peered":"2024-05-11T01:14:00.426499+0000","last_clean":"2024-05-11T01:14:00.426499+0000","last_became_active":"2024-05-11T01:13:59.434594+0000","last_became_peered":"2024-05-11T01:13:59.434594+0000","last_unstale":"2024-05-11T01:14:00.426499+0000","last_undegraded":"2024-05-11T01:14:00.426499+0000","last_fullsized":"2024-05-11T01:14:00.426499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:49:33.685964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424479+0000","last_change":"2024-05-11T01:13:59.446282+0000","last_active":"2024-05-11T01:14:00.424479+0000","last_peered":"2024-05-11T01:14:00.424479+0000","last_clean":"2024-05-11T01:14:00.424479+0000","last_became_active":"2024-05-11T01:13:59.445971+0000","last_became_peered":"2024-05-11T01:13:59.445971+0000","last_unstale":"2024-05-11T01:14:00.424479+0000","last_undegraded":"2024-05-11T01:14:00.424479+0000","last_fullsized":"2024-05-11T01:14:00.424479+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:50.869559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423979+0000","last_change":"2024-05-11T01:13:59.430661+0000","last_active":"2024-05-11T01:14:00.423979+0000","last_peered":"2024-05-11T01:14:00.423979+0000","last_clean":"2024-05-11T01:14:00.423979+0000","last_became_active":"2024-05-11T01:13:59.430408+0000","last_became_peered":"2024-05-11T01:13:59.430408+0000","last_unstale":"2024-05-11T01:14:00.423979+0000","last_undegraded":"2024-05-11T01:14:00.423979+0000","last_fullsized":"2024-05-11T01:14:00.423979+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:14:03.379607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421742+0000","last_change":"2024-05-11T01:13:59.448424+0000","last_active":"2024-05-11T01:14:00.421742+0000","last_peered":"2024-05-11T01:14:00.421742+0000","last_clean":"2024-05-11T01:14:00.421742+0000","last_became_active":"2024-05-11T01:13:59.446986+0000","last_became_peered":"2024-05-11T01:13:59.446986+0000","last_unstale":"2024-05-11T01:14:00.421742+0000","last_undegraded":"2024-05-11T01:14:00.421742+0000","last_fullsized":"2024-05-11T01:14:00.421742+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424359+0000","last_change":"2024-05-11T01:13:59.446369+0000","last_active":"2024-05-11T01:14:00.424359+0000","last_peered":"2024-05-11T01:14:00.424359+0000","last_clean":"2024-05-11T01:14:00.424359+0000","last_became_active":"2024-05-11T01:13:59.445922+0000","last_became_peered":"2024-05-11T01:13:59.445922+0000","last_unstale":"2024-05-11T01:14:00.424359+0000","last_undegraded":"2024-05-11T01:14:00.424359+0000","last_fullsized":"2024-05-11T01:14:00.424359+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:15.278361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423902+0000","last_change":"2024-05-11T01:13:59.435535+0000","last_active":"2024-05-11T01:14:00.423902+0000","last_peered":"2024-05-11T01:14:00.423902+0000","last_clean":"2024-05-11T01:14:00.423902+0000","last_became_active":"2024-05-11T01:13:59.434634+0000","last_became_peered":"2024-05-11T01:13:59.434634+0000","last_unstale":"2024-05-11T01:14:00.423902+0000","last_undegraded":"2024-05-11T01:14:00.423902+0000","last_fullsized":"2024-05-11T01:14:00.423902+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:34:16.451030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425878+0000","last_change":"2024-05-11T01:13:59.434552+0000","last_active":"2024-05-11T01:14:00.425878+0000","last_peered":"2024-05-11T01:14:00.425878+0000","last_clean":"2024-05-11T01:14:00.425878+0000","last_became_active":"2024-05-11T01:13:59.433976+0000","last_became_peered":"2024-05-11T01:13:59.433976+0000","last_unstale":"2024-05-11T01:14:00.425878+0000","last_undegraded":"2024-05-11T01:14:00.425878+0000","last_fullsized":"2024-05-11T01:14:00.425878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421319+0000","last_change":"2024-05-11T01:13:59.447658+0000","last_active":"2024-05-11T01:14:00.421319+0000","last_peered":"2024-05-11T01:14:00.421319+0000","last_clean":"2024-05-11T01:14:00.421319+0000","last_became_active":"2024-05-11T01:13:59.446507+0000","last_became_peered":"2024-05-11T01:13:59.446507+0000","last_unstale":"2024-05-11T01:14:00.421319+0000","last_undegraded":"2024-05-11T01:14:00.421319+0000","last_fullsized":"2024-05-11T01:14:00.421319+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:34:51.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421726+0000","last_change":"2024-05-11T01:13:59.449973+0000","last_active":"2024-05-11T01:14:00.421726+0000","last_peered":"2024-05-11T01:14:00.421726+0000","last_clean":"2024-05-11T01:14:00.421726+0000","last_became_active":"2024-05-11T01:13:59.449670+0000","last_became_peered":"2024-05-11T01:13:59.449670+0000","last_unstale":"2024-05-11T01:14:00.421726+0000","last_undegraded":"2024-05-11T01:14:00.421726+0000","last_fullsized":"2024-05-11T01:14:00.421726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.422340+0000","last_change":"2024-05-11T01:13:59.450134+0000","last_active":"2024-05-11T01:14:00.422340+0000","last_peered":"2024-05-11T01:14:00.422340+0000","last_clean":"2024-05-11T01:14:00.422340+0000","last_became_active":"2024-05-11T01:13:59.449822+0000","last_became_peered":"2024-05-11T01:13:59.449822+0000","last_unstale":"2024-05-11T01:14:00.422340+0000","last_undegraded":"2024-05-11T01:14:00.422340+0000","last_fullsized":"2024-05-11T01:14:00.422340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421646+0000","last_change":"2024-05-11T01:13:59.444301+0000","last_active":"2024-05-11T01:14:00.421646+0000","last_peered":"2024-05-11T01:14:00.421646+0000","last_clean":"2024-05-11T01:14:00.421646+0000","last_became_active":"2024-05-11T01:13:59.444006+0000","last_became_peered":"2024-05-11T01:13:59.444006+0000","last_unstale":"2024-05-11T01:14:00.421646+0000","last_undegraded":"2024-05-11T01:14:00.421646+0000","last_fullsized":"2024-05-11T01:14:00.421646+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:37:44.925797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425278+0000","last_change":"2024-05-11T01:13:59.428513+0000","last_active":"2024-05-11T01:14:00.425278+0000","last_peered":"2024-05-11T01:14:00.425278+0000","last_clean":"2024-05-11T01:14:00.425278+0000","last_became_active":"2024-05-11T01:13:59.428202+0000","last_became_peered":"2024-05-11T01:13:59.428202+0000","last_unstale":"2024-05-11T01:14:00.425278+0000","last_undegraded":"2024-05-11T01:14:00.425278+0000","last_fullsized":"2024-05-11T01:14:00.425278+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:23.500916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424674+0000","last_change":"2024-05-11T01:13:59.435402+0000","last_active":"2024-05-11T01:14:00.424674+0000","last_peered":"2024-05-11T01:14:00.424674+0000","last_clean":"2024-05-11T01:14:00.424674+0000","last_became_active":"2024-05-11T01:13:59.433324+0000","last_became_peered":"2024-05-11T01:13:59.433324+0000","last_unstale":"2024-05-11T01:14:00.424674+0000","last_undegraded":"2024-05-11T01:14:00.424674+0000","last_fullsized":"2024-05-11T01:14:00.424674+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:36:46.612605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421779+0000","last_change":"2024-05-11T01:13:59.447113+0000","last_active":"2024-05-11T01:14:00.421779+0000","last_peered":"2024-05-11T01:14:00.421779+0000","last_clean":"2024-05-11T01:14:00.421779+0000","last_became_active":"2024-05-11T01:13:59.446349+0000","last_became_peered":"2024-05-11T01:13:59.446349+0000","last_unstale":"2024-05-11T01:14:00.421779+0000","last_undegraded":"2024-05-11T01:14:00.421779+0000","last_fullsized":"2024-05-11T01:14:00.421779+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424048+0000","last_change":"2024-05-11T01:13:59.437811+0000","last_active":"2024-05-11T01:14:00.424048+0000","last_peered":"2024-05-11T01:14:00.424048+0000","last_clean":"2024-05-11T01:14:00.424048+0000","last_became_active":"2024-05-11T01:13:59.427975+0000","last_became_peered":"2024-05-11T01:13:59.427975+0000","last_unstale":"2024-05-11T01:14:00.424048+0000","last_undegraded":"2024-05-11T01:14:00.424048+0000","last_fullsized":"2024-05-11T01:14:00.424048+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:05:27.108182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426470+0000","last_change":"2024-05-11T01:13:59.446666+0000","last_active":"2024-05-11T01:14:00.426470+0000","last_peered":"2024-05-11T01:14:00.426470+0000","last_clean":"2024-05-11T01:14:00.426470+0000","last_became_active":"2024-05-11T01:13:59.446396+0000","last_became_peered":"2024-05-11T01:13:59.446396+0000","last_unstale":"2024-05-11T01:14:00.426470+0000","last_undegraded":"2024-05-11T01:14:00.426470+0000","last_fullsized":"2024-05-11T01:14:00.426470+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:55:23.003515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424221+0000","last_change":"2024-05-11T01:13:59.437261+0000","last_active":"2024-05-11T01:14:00.424221+0000","last_peered":"2024-05-11T01:14:00.424221+0000","last_clean":"2024-05-11T01:14:00.424221+0000","last_became_active":"2024-05-11T01:13:59.436992+0000","last_became_peered":"2024-05-11T01:13:59.436992+0000","last_unstale":"2024-05-11T01:14:00.424221+0000","last_undegraded":"2024-05-11T01:14:00.424221+0000","last_fullsized":"2024-05-11T01:14:00.424221+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:53.116569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424262+0000","last_change":"2024-05-11T01:13:59.445508+0000","last_active":"2024-05-11T01:14:00.424262+0000","last_peered":"2024-05-11T01:14:00.424262+0000","last_clean":"2024-05-11T01:14:00.424262+0000","last_became_active":"2024-05-11T01:13:59.445318+0000","last_became_peered":"2024-05-11T01:13:59.445318+0000","last_unstale":"2024-05-11T01:14:00.424262+0000","last_undegraded":"2024-05-11T01:14:00.424262+0000","last_fullsized":"2024-05-11T01:14:00.424262+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:21.154101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.427794+0000","last_change":"2024-05-11T01:13:59.446453+0000","last_active":"2024-05-11T01:14:00.427794+0000","last_peered":"2024-05-11T01:14:00.427794+0000","last_clean":"2024-05-11T01:14:00.427794+0000","last_became_active":"2024-05-11T01:13:59.446160+0000","last_became_peered":"2024-05-11T01:13:59.446160+0000","last_unstale":"2024-05-11T01:14:00.427794+0000","last_undegraded":"2024-05-11T01:14:00.427794+0000","last_fullsized":"2024-05-11T01:14:00.427794+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:32:54.808774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424632+0000","last_change":"2024-05-11T01:13:59.444231+0000","last_active":"2024-05-11T01:14:00.424632+0000","last_peered":"2024-05-11T01:14:00.424632+0000","last_clean":"2024-05-11T01:14:00.424632+0000","last_became_active":"2024-05-11T01:13:59.443762+0000","last_became_peered":"2024-05-11T01:13:59.443762+0000","last_unstale":"2024-05-11T01:14:00.424632+0000","last_undegraded":"2024-05-11T01:14:00.424632+0000","last_fullsized":"2024-05-11T01:14:00.424632+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:25:17.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424359+0000","last_change":"2024-05-11T01:13:59.437781+0000","last_active":"2024-05-11T01:14:00.424359+0000","last_peered":"2024-05-11T01:14:00.424359+0000","last_clean":"2024-05-11T01:14:00.424359+0000","last_became_active":"2024-05-11T01:13:59.437444+0000","last_became_peered":"2024-05-11T01:13:59.437444+0000","last_unstale":"2024-05-11T01:14:00.424359+0000","last_undegraded":"2024-05-11T01:14:00.424359+0000","last_fullsized":"2024-05-11T01:14:00.424359+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:31:12.326102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421934+0000","last_change":"2024-05-11T01:13:59.447343+0000","last_active":"2024-05-11T01:14:00.421934+0000","last_peered":"2024-05-11T01:14:00.421934+0000","last_clean":"2024-05-11T01:14:00.421934+0000","last_became_active":"2024-05-11T01:13:59.446779+0000","last_became_peered":"2024-05-11T01:13:59.446779+0000","last_unstale":"2024-05-11T01:14:00.421934+0000","last_undegraded":"2024-05-11T01:14:00.421934+0000","last_fullsized":"2024-05-11T01:14:00.421934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.427937+0000","last_change":"2024-05-11T01:13:59.431582+0000","last_active":"2024-05-11T01:14:00.427937+0000","last_peered":"2024-05-11T01:14:00.427937+0000","last_clean":"2024-05-11T01:14:00.427937+0000","last_became_active":"2024-05-11T01:13:59.431309+0000","last_became_peered":"2024-05-11T01:13:59.431309+0000","last_unstale":"2024-05-11T01:14:00.427937+0000","last_undegraded":"2024-05-11T01:14:00.427937+0000","last_fullsized":"2024-05-11T01:14:00.427937+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426597+0000","last_change":"2024-05-11T01:13:59.433902+0000","last_active":"2024-05-11T01:14:00.426597+0000","last_peered":"2024-05-11T01:14:00.426597+0000","last_clean":"2024-05-11T01:14:00.426597+0000","last_became_active":"2024-05-11T01:13:59.433606+0000","last_became_peered":"2024-05-11T01:13:59.433606+0000","last_unstale":"2024-05-11T01:14:00.426597+0000","last_undegraded":"2024-05-11T01:14:00.426597+0000","last_fullsized":"2024-05-11T01:14:00.426597+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.427882+0000","last_change":"2024-05-11T01:13:59.436026+0000","last_active":"2024-05-11T01:14:00.427882+0000","last_peered":"2024-05-11T01:14:00.427882+0000","last_clean":"2024-05-11T01:14:00.427882+0000","last_became_active":"2024-05-11T01:13:59.435708+0000","last_became_peered":"2024-05-11T01:13:59.435708+0000","last_unstale":"2024-05-11T01:14:00.427882+0000","last_undegraded":"2024-05-11T01:14:00.427882+0000","last_fullsized":"2024-05-11T01:14:00.427882+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424031+0000","last_change":"2024-05-11T01:13:59.441132+0000","last_active":"2024-05-11T01:14:00.424031+0000","last_peered":"2024-05-11T01:14:00.424031+0000","last_clean":"2024-05-11T01:14:00.424031+0000","last_became_active":"2024-05-11T01:13:59.440852+0000","last_became_peered":"2024-05-11T01:13:59.440852+0000","last_unstale":"2024-05-11T01:14:00.424031+0000","last_undegraded":"2024-05-11T01:14:00.424031+0000","last_fullsized":"2024-05-11T01:14:00.424031+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:18:58.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424381+0000","last_change":"2024-05-11T01:13:59.436547+0000","last_active":"2024-05-11T01:14:00.424381+0000","last_peered":"2024-05-11T01:14:00.424381+0000","last_clean":"2024-05-11T01:14:00.424381+0000","last_became_active":"2024-05-11T01:13:59.436260+0000","last_became_peered":"2024-05-11T01:13:59.436260+0000","last_unstale":"2024-05-11T01:14:00.424381+0000","last_undegraded":"2024-05-11T01:14:00.424381+0000","last_fullsized":"2024-05-11T01:14:00.424381+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:27.176219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424820+0000","last_change":"2024-05-11T01:13:59.425177+0000","last_active":"2024-05-11T01:14:00.424820+0000","last_peered":"2024-05-11T01:14:00.424820+0000","last_clean":"2024-05-11T01:14:00.424820+0000","last_became_active":"2024-05-11T01:13:59.424328+0000","last_became_peered":"2024-05-11T01:13:59.424328+0000","last_unstale":"2024-05-11T01:14:00.424820+0000","last_undegraded":"2024-05-11T01:14:00.424820+0000","last_fullsized":"2024-05-11T01:14:00.424820+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:29.739616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426743+0000","last_change":"2024-05-11T01:13:59.436551+0000","last_active":"2024-05-11T01:14:00.426743+0000","last_peered":"2024-05-11T01:14:00.426743+0000","last_clean":"2024-05-11T01:14:00.426743+0000","last_became_active":"2024-05-11T01:13:59.436257+0000","last_became_peered":"2024-05-11T01:13:59.436257+0000","last_unstale":"2024-05-11T01:14:00.426743+0000","last_undegraded":"2024-05-11T01:14:00.426743+0000","last_fullsized":"2024-05-11T01:14:00.426743+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:37:44.925797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423903+0000","last_change":"2024-05-11T01:13:59.435398+0000","last_active":"2024-05-11T01:14:00.423903+0000","last_peered":"2024-05-11T01:14:00.423903+0000","last_clean":"2024-05-11T01:14:00.423903+0000","last_became_active":"2024-05-11T01:13:59.435148+0000","last_became_peered":"2024-05-11T01:13:59.435148+0000","last_unstale":"2024-05-11T01:14:00.423903+0000","last_undegraded":"2024-05-11T01:14:00.423903+0000","last_fullsized":"2024-05-11T01:14:00.423903+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:14:43.435242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424198+0000","last_change":"2024-05-11T01:13:59.438524+0000","last_active":"2024-05-11T01:14:00.424198+0000","last_peered":"2024-05-11T01:14:00.424198+0000","last_clean":"2024-05-11T01:14:00.424198+0000","last_became_active":"2024-05-11T01:13:59.438262+0000","last_became_peered":"2024-05-11T01:13:59.438262+0000","last_unstale":"2024-05-11T01:14:00.424198+0000","last_undegraded":"2024-05-11T01:14:00.424198+0000","last_fullsized":"2024-05-11T01:14:00.424198+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:44:46.930438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424240+0000","last_change":"2024-05-11T01:13:59.432090+0000","last_active":"2024-05-11T01:14:00.424240+0000","last_peered":"2024-05-11T01:14:00.424240+0000","last_clean":"2024-05-11T01:14:00.424240+0000","last_became_active":"2024-05-11T01:13:59.431691+0000","last_became_peered":"2024-05-11T01:13:59.431691+0000","last_unstale":"2024-05-11T01:14:00.424240+0000","last_undegraded":"2024-05-11T01:14:00.424240+0000","last_fullsized":"2024-05-11T01:14:00.424240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:11:10.857350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425306+0000","last_change":"2024-05-11T01:13:59.440442+0000","last_active":"2024-05-11T01:14:00.425306+0000","last_peered":"2024-05-11T01:14:00.425306+0000","last_clean":"2024-05-11T01:14:00.425306+0000","last_became_active":"2024-05-11T01:13:59.440107+0000","last_became_peered":"2024-05-11T01:13:59.440107+0000","last_unstale":"2024-05-11T01:14:00.425306+0000","last_undegraded":"2024-05-11T01:14:00.425306+0000","last_fullsized":"2024-05-11T01:14:00.425306+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:58:40.948977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424212+0000","last_change":"2024-05-11T01:13:58.440158+0000","last_active":"2024-05-11T01:14:00.424212+0000","last_peered":"2024-05-11T01:14:00.424212+0000","last_clean":"2024-05-11T01:14:00.424212+0000","last_became_active":"2024-05-11T01:13:58.439841+0000","last_became_peered":"2024-05-11T01:13:58.439841+0000","last_unstale":"2024-05-11T01:14:00.424212+0000","last_undegraded":"2024-05-11T01:14:00.424212+0000","last_fullsized":"2024-05-11T01:14:00.424212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.427889+0000","last_change":"2024-05-11T01:13:59.433723+0000","last_active":"2024-05-11T01:14:00.427889+0000","last_peered":"2024-05-11T01:14:00.427889+0000","last_clean":"2024-05-11T01:14:00.427889+0000","last_became_active":"2024-05-11T01:13:59.433441+0000","last_became_peered":"2024-05-11T01:13:59.433441+0000","last_unstale":"2024-05-11T01:14:00.427889+0000","last_undegraded":"2024-05-11T01:14:00.427889+0000","last_fullsized":"2024-05-11T01:14:00.427889+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424500+0000","last_change":"2024-05-11T01:13:58.433974+0000","last_active":"2024-05-11T01:14:00.424500+0000","last_peered":"2024-05-11T01:14:00.424500+0000","last_clean":"2024-05-11T01:14:00.424500+0000","last_became_active":"2024-05-11T01:13:58.433649+0000","last_became_peered":"2024-05-11T01:13:58.433649+0000","last_unstale":"2024-05-11T01:14:00.424500+0000","last_undegraded":"2024-05-11T01:14:00.424500+0000","last_fullsized":"2024-05-11T01:14:00.424500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:59.322532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424977+0000","last_change":"2024-05-11T01:13:59.439364+0000","last_active":"2024-05-11T01:14:00.424977+0000","last_peered":"2024-05-11T01:14:00.424977+0000","last_clean":"2024-05-11T01:14:00.424977+0000","last_became_active":"2024-05-11T01:13:59.439073+0000","last_became_peered":"2024-05-11T01:13:59.439073+0000","last_unstale":"2024-05-11T01:14:00.424977+0000","last_undegraded":"2024-05-11T01:14:00.424977+0000","last_fullsized":"2024-05-11T01:14:00.424977+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:48.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425486+0000","last_change":"2024-05-11T01:13:58.435449+0000","last_active":"2024-05-11T01:14:00.425486+0000","last_peered":"2024-05-11T01:14:00.425486+0000","last_clean":"2024-05-11T01:14:00.425486+0000","last_became_active":"2024-05-11T01:13:58.434345+0000","last_became_peered":"2024-05-11T01:13:58.434345+0000","last_unstale":"2024-05-11T01:14:00.425486+0000","last_undegraded":"2024-05-11T01:14:00.425486+0000","last_fullsized":"2024-05-11T01:14:00.425486+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:08:58.718268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425493+0000","last_change":"2024-05-11T01:13:59.440315+0000","last_active":"2024-05-11T01:14:00.425493+0000","last_peered":"2024-05-11T01:14:00.425493+0000","last_clean":"2024-05-11T01:14:00.425493+0000","last_became_active":"2024-05-11T01:13:59.439153+0000","last_became_peered":"2024-05-11T01:13:59.439153+0000","last_unstale":"2024-05-11T01:14:00.425493+0000","last_undegraded":"2024-05-11T01:14:00.425493+0000","last_fullsized":"2024-05-11T01:14:00.425493+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:38:04.924508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425916+0000","last_change":"2024-05-11T01:13:58.434357+0000","last_active":"2024-05-11T01:14:00.425916+0000","last_peered":"2024-05-11T01:14:00.425916+0000","last_clean":"2024-05-11T01:14:00.425916+0000","last_became_active":"2024-05-11T01:13:58.433392+0000","last_became_peered":"2024-05-11T01:13:58.433392+0000","last_unstale":"2024-05-11T01:14:00.425916+0000","last_undegraded":"2024-05-11T01:14:00.425916+0000","last_fullsized":"2024-05-11T01:14:00.425916+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:14.276107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424274+0000","last_change":"2024-05-11T01:13:59.435145+0000","last_active":"2024-05-11T01:14:00.424274+0000","last_peered":"2024-05-11T01:14:00.424274+0000","last_clean":"2024-05-11T01:14:00.424274+0000","last_became_active":"2024-05-11T01:13:59.434862+0000","last_became_peered":"2024-05-11T01:13:59.434862+0000","last_unstale":"2024-05-11T01:14:00.424274+0000","last_undegraded":"2024-05-11T01:14:00.424274+0000","last_fullsized":"2024-05-11T01:14:00.424274+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:17.395853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424640+0000","last_change":"2024-05-11T01:13:58.439588+0000","last_active":"2024-05-11T01:14:00.424640+0000","last_peered":"2024-05-11T01:14:00.424640+0000","last_clean":"2024-05-11T01:14:00.424640+0000","last_became_active":"2024-05-11T01:13:58.439133+0000","last_became_peered":"2024-05-11T01:13:58.439133+0000","last_unstale":"2024-05-11T01:14:00.424640+0000","last_undegraded":"2024-05-11T01:14:00.424640+0000","last_fullsized":"2024-05-11T01:14:00.424640+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421983+0000","last_change":"2024-05-11T01:13:59.448882+0000","last_active":"2024-05-11T01:14:00.421983+0000","last_peered":"2024-05-11T01:14:00.421983+0000","last_clean":"2024-05-11T01:14:00.421983+0000","last_became_active":"2024-05-11T01:13:59.448208+0000","last_became_peered":"2024-05-11T01:13:59.448208+0000","last_unstale":"2024-05-11T01:14:00.421983+0000","last_undegraded":"2024-05-11T01:14:00.421983+0000","last_fullsized":"2024-05-11T01:14:00.421983+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424024+0000","last_change":"2024-05-11T01:13:58.428838+0000","last_active":"2024-05-11T01:14:00.424024+0000","last_peered":"2024-05-11T01:14:00.424024+0000","last_clean":"2024-05-11T01:14:00.424024+0000","last_became_active":"2024-05-11T01:13:58.428556+0000","last_became_peered":"2024-05-11T01:13:58.428556+0000","last_unstale":"2024-05-11T01:14:00.424024+0000","last_undegraded":"2024-05-11T01:14:00.424024+0000","last_fullsized":"2024-05-11T01:14:00.424024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:01:35.971417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421389+0000","last_change":"2024-05-11T01:13:59.447010+0000","last_active":"2024-05-11T01:14:00.421389+0000","last_peered":"2024-05-11T01:14:00.421389+0000","last_clean":"2024-05-11T01:14:00.421389+0000","last_became_active":"2024-05-11T01:13:59.446713+0000","last_became_peered":"2024-05-11T01:13:59.446713+0000","last_unstale":"2024-05-11T01:14:00.421389+0000","last_undegraded":"2024-05-11T01:14:00.421389+0000","last_fullsized":"2024-05-11T01:14:00.421389+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:03:04.844818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424193+0000","last_change":"2024-05-11T01:13:58.428031+0000","last_active":"2024-05-11T01:14:00.424193+0000","last_peered":"2024-05-11T01:14:00.424193+0000","last_clean":"2024-05-11T01:14:00.424193+0000","last_became_active":"2024-05-11T01:13:58.427742+0000","last_became_peered":"2024-05-11T01:13:58.427742+0000","last_unstale":"2024-05-11T01:14:00.424193+0000","last_undegraded":"2024-05-11T01:14:00.424193+0000","last_fullsized":"2024-05-11T01:14:00.424193+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.427756+0000","last_change":"2024-05-11T01:13:59.431589+0000","last_active":"2024-05-11T01:14:00.427756+0000","last_peered":"2024-05-11T01:14:00.427756+0000","last_clean":"2024-05-11T01:14:00.427756+0000","last_became_active":"2024-05-11T01:13:59.431292+0000","last_became_peered":"2024-05-11T01:13:59.431292+0000","last_unstale":"2024-05-11T01:14:00.427756+0000","last_undegraded":"2024-05-11T01:14:00.427756+0000","last_fullsized":"2024-05-11T01:14:00.427756+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421412+0000","last_change":"2024-05-11T01:13:58.436246+0000","last_active":"2024-05-11T01:14:00.421412+0000","last_peered":"2024-05-11T01:14:00.421412+0000","last_clean":"2024-05-11T01:14:00.421412+0000","last_became_active":"2024-05-11T01:13:58.435967+0000","last_became_peered":"2024-05-11T01:13:58.435967+0000","last_unstale":"2024-05-11T01:14:00.421412+0000","last_undegraded":"2024-05-11T01:14:00.421412+0000","last_fullsized":"2024-05-11T01:14:00.421412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:22.242555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424432+0000","last_change":"2024-05-11T01:13:59.445506+0000","last_active":"2024-05-11T01:14:00.424432+0000","last_peered":"2024-05-11T01:14:00.424432+0000","last_clean":"2024-05-11T01:14:00.424432+0000","last_became_active":"2024-05-11T01:13:59.445232+0000","last_became_peered":"2024-05-11T01:13:59.445232+0000","last_unstale":"2024-05-11T01:14:00.424432+0000","last_undegraded":"2024-05-11T01:14:00.424432+0000","last_fullsized":"2024-05-11T01:14:00.424432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:18:27.371597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424424+0000","last_change":"2024-05-11T01:13:58.426187+0000","last_active":"2024-05-11T01:14:00.424424+0000","last_peered":"2024-05-11T01:14:00.424424+0000","last_clean":"2024-05-11T01:14:00.424424+0000","last_became_active":"2024-05-11T01:13:58.425913+0000","last_became_peered":"2024-05-11T01:13:58.425913+0000","last_unstale":"2024-05-11T01:14:00.424424+0000","last_undegraded":"2024-05-11T01:14:00.424424+0000","last_fullsized":"2024-05-11T01:14:00.424424+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426499+0000","last_change":"2024-05-11T01:13:59.431401+0000","last_active":"2024-05-11T01:14:00.426499+0000","last_peered":"2024-05-11T01:14:00.426499+0000","last_clean":"2024-05-11T01:14:00.426499+0000","last_became_active":"2024-05-11T01:13:59.431126+0000","last_became_peered":"2024-05-11T01:13:59.431126+0000","last_unstale":"2024-05-11T01:14:00.426499+0000","last_undegraded":"2024-05-11T01:14:00.426499+0000","last_fullsized":"2024-05-11T01:14:00.426499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:42:47.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.422678+0000","last_change":"2024-05-11T01:13:58.431417+0000","last_active":"2024-05-11T01:14:00.422678+0000","last_peered":"2024-05-11T01:14:00.422678+0000","last_clean":"2024-05-11T01:14:00.422678+0000","last_became_active":"2024-05-11T01:13:58.431132+0000","last_became_peered":"2024-05-11T01:13:58.431132+0000","last_unstale":"2024-05-11T01:14:00.422678+0000","last_undegraded":"2024-05-11T01:14:00.422678+0000","last_fullsized":"2024-05-11T01:14:00.422678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:23:16.977274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424523+0000","last_change":"2024-05-11T01:13:59.436396+0000","last_active":"2024-05-11T01:14:00.424523+0000","last_peered":"2024-05-11T01:14:00.424523+0000","last_clean":"2024-05-11T01:14:00.424523+0000","last_became_active":"2024-05-11T01:13:59.436043+0000","last_became_peered":"2024-05-11T01:13:59.436043+0000","last_unstale":"2024-05-11T01:14:00.424523+0000","last_undegraded":"2024-05-11T01:14:00.424523+0000","last_fullsized":"2024-05-11T01:14:00.424523+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426232+0000","last_change":"2024-05-11T01:13:58.435566+0000","last_active":"2024-05-11T01:14:00.426232+0000","last_peered":"2024-05-11T01:14:00.426232+0000","last_clean":"2024-05-11T01:14:00.426232+0000","last_became_active":"2024-05-11T01:13:58.435280+0000","last_became_peered":"2024-05-11T01:13:58.435280+0000","last_unstale":"2024-05-11T01:14:00.426232+0000","last_undegraded":"2024-05-11T01:14:00.426232+0000","last_fullsized":"2024-05-11T01:14:00.426232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:47.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425975+0000","last_change":"2024-05-11T01:13:59.432307+0000","last_active":"2024-05-11T01:14:00.425975+0000","last_peered":"2024-05-11T01:14:00.425975+0000","last_clean":"2024-05-11T01:14:00.425975+0000","last_became_active":"2024-05-11T01:13:59.431913+0000","last_became_peered":"2024-05-11T01:13:59.431913+0000","last_unstale":"2024-05-11T01:14:00.425975+0000","last_undegraded":"2024-05-11T01:14:00.425975+0000","last_fullsized":"2024-05-11T01:14:00.425975+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"60'6","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424733+0000","last_change":"2024-05-11T01:13:58.425157+0000","last_active":"2024-05-11T01:14:00.424733+0000","last_peered":"2024-05-11T01:14:00.424733+0000","last_clean":"2024-05-11T01:14:00.424733+0000","last_became_active":"2024-05-11T01:13:58.424894+0000","last_became_peered":"2024-05-11T01:13:58.424894+0000","last_unstale":"2024-05-11T01:14:00.424733+0000","last_undegraded":"2024-05-11T01:14:00.424733+0000","last_fullsized":"2024-05-11T01:14:00.424733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:48:11.424880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.427437+0000","last_change":"2024-05-11T01:13:59.447362+0000","last_active":"2024-05-11T01:14:00.427437+0000","last_peered":"2024-05-11T01:14:00.427437+0000","last_clean":"2024-05-11T01:14:00.427437+0000","last_became_active":"2024-05-11T01:13:59.447082+0000","last_became_peered":"2024-05-11T01:13:59.447082+0000","last_unstale":"2024-05-11T01:14:00.427437+0000","last_undegraded":"2024-05-11T01:14:00.427437+0000","last_fullsized":"2024-05-11T01:14:00.427437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424741+0000","last_change":"2024-05-11T01:13:58.439044+0000","last_active":"2024-05-11T01:14:00.424741+0000","last_peered":"2024-05-11T01:14:00.424741+0000","last_clean":"2024-05-11T01:14:00.424741+0000","last_became_active":"2024-05-11T01:13:58.438603+0000","last_became_peered":"2024-05-11T01:13:58.438603+0000","last_unstale":"2024-05-11T01:14:00.424741+0000","last_undegraded":"2024-05-11T01:14:00.424741+0000","last_fullsized":"2024-05-11T01:14:00.424741+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421937+0000","last_change":"2024-05-11T01:13:59.449114+0000","last_active":"2024-05-11T01:14:00.421937+0000","last_peered":"2024-05-11T01:14:00.421937+0000","last_clean":"2024-05-11T01:14:00.421937+0000","last_became_active":"2024-05-11T01:13:59.448532+0000","last_became_peered":"2024-05-11T01:13:59.448532+0000","last_unstale":"2024-05-11T01:14:00.421937+0000","last_undegraded":"2024-05-11T01:14:00.421937+0000","last_fullsized":"2024-05-11T01:14:00.421937+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425738+0000","last_change":"2024-05-11T01:13:58.434118+0000","last_active":"2024-05-11T01:14:00.425738+0000","last_peered":"2024-05-11T01:14:00.425738+0000","last_clean":"2024-05-11T01:14:00.425738+0000","last_became_active":"2024-05-11T01:13:58.433818+0000","last_became_peered":"2024-05-11T01:13:58.433818+0000","last_unstale":"2024-05-11T01:14:00.425738+0000","last_undegraded":"2024-05-11T01:14:00.425738+0000","last_fullsized":"2024-05-11T01:14:00.425738+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:49.867304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423798+0000","last_change":"2024-05-11T01:13:59.435013+0000","last_active":"2024-05-11T01:14:00.423798+0000","last_peered":"2024-05-11T01:14:00.423798+0000","last_clean":"2024-05-11T01:14:00.423798+0000","last_became_active":"2024-05-11T01:13:59.434726+0000","last_became_peered":"2024-05-11T01:13:59.434726+0000","last_unstale":"2024-05-11T01:14:00.423798+0000","last_undegraded":"2024-05-11T01:14:00.423798+0000","last_fullsized":"2024-05-11T01:14:00.423798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:03:22.763694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424136+0000","last_change":"2024-05-11T01:13:58.438894+0000","last_active":"2024-05-11T01:14:00.424136+0000","last_peered":"2024-05-11T01:14:00.424136+0000","last_clean":"2024-05-11T01:14:00.424136+0000","last_became_active":"2024-05-11T01:13:58.438528+0000","last_became_peered":"2024-05-11T01:13:58.438528+0000","last_unstale":"2024-05-11T01:14:00.424136+0000","last_undegraded":"2024-05-11T01:14:00.424136+0000","last_fullsized":"2024-05-11T01:14:00.424136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426033+0000","last_change":"2024-05-11T01:13:59.437777+0000","last_active":"2024-05-11T01:14:00.426033+0000","last_peered":"2024-05-11T01:14:00.426033+0000","last_clean":"2024-05-11T01:14:00.426033+0000","last_became_active":"2024-05-11T01:13:59.437176+0000","last_became_peered":"2024-05-11T01:13:59.437176+0000","last_unstale":"2024-05-11T01:14:00.426033+0000","last_undegraded":"2024-05-11T01:14:00.426033+0000","last_fullsized":"2024-05-11T01:14:00.426033+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:07:21.319542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421494+0000","last_change":"2024-05-11T01:13:58.431218+0000","last_active":"2024-05-11T01:14:00.421494+0000","last_peered":"2024-05-11T01:14:00.421494+0000","last_clean":"2024-05-11T01:14:00.421494+0000","last_became_active":"2024-05-11T01:13:58.430952+0000","last_became_peered":"2024-05-11T01:13:58.430952+0000","last_unstale":"2024-05-11T01:14:00.421494+0000","last_undegraded":"2024-05-11T01:14:00.421494+0000","last_fullsized":"2024-05-11T01:14:00.421494+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:03:39.335329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424175+0000","last_change":"2024-05-11T01:13:59.427029+0000","last_active":"2024-05-11T01:14:00.424175+0000","last_peered":"2024-05-11T01:14:00.424175+0000","last_clean":"2024-05-11T01:14:00.424175+0000","last_became_active":"2024-05-11T01:13:59.426727+0000","last_became_peered":"2024-05-11T01:13:59.426727+0000","last_unstale":"2024-05-11T01:14:00.424175+0000","last_undegraded":"2024-05-11T01:14:00.424175+0000","last_fullsized":"2024-05-11T01:14:00.424175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:52.626350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424360+0000","last_change":"2024-05-11T01:13:58.434507+0000","last_active":"2024-05-11T01:14:00.424360+0000","last_peered":"2024-05-11T01:14:00.424360+0000","last_clean":"2024-05-11T01:14:00.424360+0000","last_became_active":"2024-05-11T01:13:58.434215+0000","last_became_peered":"2024-05-11T01:13:58.434215+0000","last_unstale":"2024-05-11T01:14:00.424360+0000","last_undegraded":"2024-05-11T01:14:00.424360+0000","last_fullsized":"2024-05-11T01:14:00.424360+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:35.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421887+0000","last_change":"2024-05-11T01:13:59.444388+0000","last_active":"2024-05-11T01:14:00.421887+0000","last_peered":"2024-05-11T01:14:00.421887+0000","last_clean":"2024-05-11T01:14:00.421887+0000","last_became_active":"2024-05-11T01:13:59.443981+0000","last_became_peered":"2024-05-11T01:13:59.443981+0000","last_unstale":"2024-05-11T01:14:00.421887+0000","last_undegraded":"2024-05-11T01:14:00.421887+0000","last_fullsized":"2024-05-11T01:14:00.421887+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425048+0000","last_change":"2024-05-11T01:13:58.429801+0000","last_active":"2024-05-11T01:14:00.425048+0000","last_peered":"2024-05-11T01:14:00.425048+0000","last_clean":"2024-05-11T01:14:00.425048+0000","last_became_active":"2024-05-11T01:13:58.429538+0000","last_became_peered":"2024-05-11T01:13:58.429538+0000","last_unstale":"2024-05-11T01:14:00.425048+0000","last_undegraded":"2024-05-11T01:14:00.425048+0000","last_fullsized":"2024-05-11T01:14:00.425048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:27.314762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.427156+0000","last_change":"2024-05-11T01:13:59.438763+0000","last_active":"2024-05-11T01:14:00.427156+0000","last_peered":"2024-05-11T01:14:00.427156+0000","last_clean":"2024-05-11T01:14:00.427156+0000","last_became_active":"2024-05-11T01:13:59.438474+0000","last_became_peered":"2024-05-11T01:13:59.438474+0000","last_unstale":"2024-05-11T01:14:00.427156+0000","last_undegraded":"2024-05-11T01:14:00.427156+0000","last_fullsized":"2024-05-11T01:14:00.427156+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424362+0000","last_change":"2024-05-11T01:13:58.426925+0000","last_active":"2024-05-11T01:14:00.424362+0000","last_peered":"2024-05-11T01:14:00.424362+0000","last_clean":"2024-05-11T01:14:00.424362+0000","last_became_active":"2024-05-11T01:13:58.426586+0000","last_became_peered":"2024-05-11T01:13:58.426586+0000","last_unstale":"2024-05-11T01:14:00.424362+0000","last_undegraded":"2024-05-11T01:14:00.424362+0000","last_fullsized":"2024-05-11T01:14:00.424362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.427715+0000","last_change":"2024-05-11T01:13:59.436970+0000","last_active":"2024-05-11T01:14:00.427715+0000","last_peered":"2024-05-11T01:14:00.427715+0000","last_clean":"2024-05-11T01:14:00.427715+0000","last_became_active":"2024-05-11T01:13:59.436373+0000","last_became_peered":"2024-05-11T01:13:59.436373+0000","last_unstale":"2024-05-11T01:14:00.427715+0000","last_undegraded":"2024-05-11T01:14:00.427715+0000","last_fullsized":"2024-05-11T01:14:00.427715+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424523+0000","last_change":"2024-05-11T01:13:58.439680+0000","last_active":"2024-05-11T01:14:00.424523+0000","last_peered":"2024-05-11T01:14:00.424523+0000","last_clean":"2024-05-11T01:14:00.424523+0000","last_became_active":"2024-05-11T01:13:58.439404+0000","last_became_peered":"2024-05-11T01:13:58.439404+0000","last_unstale":"2024-05-11T01:14:00.424523+0000","last_undegraded":"2024-05-11T01:14:00.424523+0000","last_fullsized":"2024-05-11T01:14:00.424523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:41:26.236385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424081+0000","last_change":"2024-05-11T01:13:59.435753+0000","last_active":"2024-05-11T01:14:00.424081+0000","last_peered":"2024-05-11T01:14:00.424081+0000","last_clean":"2024-05-11T01:14:00.424081+0000","last_became_active":"2024-05-11T01:13:59.435253+0000","last_became_peered":"2024-05-11T01:13:59.435253+0000","last_unstale":"2024-05-11T01:14:00.424081+0000","last_undegraded":"2024-05-11T01:14:00.424081+0000","last_fullsized":"2024-05-11T01:14:00.424081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:06:58.758258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424848+0000","last_change":"2024-05-11T01:13:58.437862+0000","last_active":"2024-05-11T01:14:00.424848+0000","last_peered":"2024-05-11T01:14:00.424848+0000","last_clean":"2024-05-11T01:14:00.424848+0000","last_became_active":"2024-05-11T01:13:58.437597+0000","last_became_peered":"2024-05-11T01:13:58.437597+0000","last_unstale":"2024-05-11T01:14:00.424848+0000","last_undegraded":"2024-05-11T01:14:00.424848+0000","last_fullsized":"2024-05-11T01:14:00.424848+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:19:23.818943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425229+0000","last_change":"2024-05-11T01:13:59.431673+0000","last_active":"2024-05-11T01:14:00.425229+0000","last_peered":"2024-05-11T01:14:00.425229+0000","last_clean":"2024-05-11T01:14:00.425229+0000","last_became_active":"2024-05-11T01:13:59.431386+0000","last_became_peered":"2024-05-11T01:13:59.431386+0000","last_unstale":"2024-05-11T01:14:00.425229+0000","last_undegraded":"2024-05-11T01:14:00.425229+0000","last_fullsized":"2024-05-11T01:14:00.425229+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:27.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424097+0000","last_change":"2024-05-11T01:13:58.431189+0000","last_active":"2024-05-11T01:14:00.424097+0000","last_peered":"2024-05-11T01:14:00.424097+0000","last_clean":"2024-05-11T01:14:00.424097+0000","last_became_active":"2024-05-11T01:13:58.430890+0000","last_became_peered":"2024-05-11T01:13:58.430890+0000","last_unstale":"2024-05-11T01:14:00.424097+0000","last_undegraded":"2024-05-11T01:14:00.424097+0000","last_fullsized":"2024-05-11T01:14:00.424097+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424319+0000","last_change":"2024-05-11T01:13:59.429997+0000","last_active":"2024-05-11T01:14:00.424319+0000","last_peered":"2024-05-11T01:14:00.424319+0000","last_clean":"2024-05-11T01:14:00.424319+0000","last_became_active":"2024-05-11T01:13:59.429734+0000","last_became_peered":"2024-05-11T01:13:59.429734+0000","last_unstale":"2024-05-11T01:14:00.424319+0000","last_undegraded":"2024-05-11T01:14:00.424319+0000","last_fullsized":"2024-05-11T01:14:00.424319+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:47.445191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421543+0000","last_change":"2024-05-11T01:13:58.431136+0000","last_active":"2024-05-11T01:14:00.421543+0000","last_peered":"2024-05-11T01:14:00.421543+0000","last_clean":"2024-05-11T01:14:00.421543+0000","last_became_active":"2024-05-11T01:13:58.430846+0000","last_became_peered":"2024-05-11T01:13:58.430846+0000","last_unstale":"2024-05-11T01:14:00.421543+0000","last_undegraded":"2024-05-11T01:14:00.421543+0000","last_fullsized":"2024-05-11T01:14:00.421543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:38.439492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421440+0000","last_change":"2024-05-11T01:13:59.450616+0000","last_active":"2024-05-11T01:14:00.421440+0000","last_peered":"2024-05-11T01:14:00.421440+0000","last_clean":"2024-05-11T01:14:00.421440+0000","last_became_active":"2024-05-11T01:13:59.450300+0000","last_became_peered":"2024-05-11T01:13:59.450300+0000","last_unstale":"2024-05-11T01:14:00.421440+0000","last_undegraded":"2024-05-11T01:14:00.421440+0000","last_fullsized":"2024-05-11T01:14:00.421440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425506+0000","last_change":"2024-05-11T01:13:58.436187+0000","last_active":"2024-05-11T01:14:00.425506+0000","last_peered":"2024-05-11T01:14:00.425506+0000","last_clean":"2024-05-11T01:14:00.425506+0000","last_became_active":"2024-05-11T01:13:58.434702+0000","last_became_peered":"2024-05-11T01:13:58.434702+0000","last_unstale":"2024-05-11T01:14:00.425506+0000","last_undegraded":"2024-05-11T01:14:00.425506+0000","last_fullsized":"2024-05-11T01:14:00.425506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:28.737362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.428020+0000","last_change":"2024-05-11T01:13:59.452028+0000","last_active":"2024-05-11T01:14:00.428020+0000","last_peered":"2024-05-11T01:14:00.428020+0000","last_clean":"2024-05-11T01:14:00.428020+0000","last_became_active":"2024-05-11T01:13:59.451773+0000","last_became_peered":"2024-05-11T01:13:59.451773+0000","last_unstale":"2024-05-11T01:14:00.428020+0000","last_undegraded":"2024-05-11T01:14:00.428020+0000","last_fullsized":"2024-05-11T01:14:00.428020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"61'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.428543+0000","last_change":"2024-05-11T01:13:58.436921+0000","last_active":"2024-05-11T01:14:00.428543+0000","last_peered":"2024-05-11T01:14:00.428543+0000","last_clean":"2024-05-11T01:14:00.428543+0000","last_became_active":"2024-05-11T01:13:58.436650+0000","last_became_peered":"2024-05-11T01:13:58.436650+0000","last_unstale":"2024-05-11T01:14:00.428543+0000","last_undegraded":"2024-05-11T01:14:00.428543+0000","last_fullsized":"2024-05-11T01:14:00.428543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:16:06.268571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421915+0000","last_change":"2024-05-11T01:13:59.442952+0000","last_active":"2024-05-11T01:14:00.421915+0000","last_peered":"2024-05-11T01:14:00.421915+0000","last_clean":"2024-05-11T01:14:00.421915+0000","last_became_active":"2024-05-11T01:13:59.442413+0000","last_became_peered":"2024-05-11T01:13:59.442413+0000","last_unstale":"2024-05-11T01:14:00.421915+0000","last_undegraded":"2024-05-11T01:14:00.421915+0000","last_fullsized":"2024-05-11T01:14:00.421915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:32:54.808774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426328+0000","last_change":"2024-05-11T01:13:58.436342+0000","last_active":"2024-05-11T01:14:00.426328+0000","last_peered":"2024-05-11T01:14:00.426328+0000","last_clean":"2024-05-11T01:14:00.426328+0000","last_became_active":"2024-05-11T01:13:58.436037+0000","last_became_peered":"2024-05-11T01:13:58.436037+0000","last_unstale":"2024-05-11T01:14:00.426328+0000","last_undegraded":"2024-05-11T01:14:00.426328+0000","last_fullsized":"2024-05-11T01:14:00.426328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:22.498662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.426553+0000","last_change":"2024-05-11T01:13:59.434239+0000","last_active":"2024-05-11T01:14:00.426553+0000","last_peered":"2024-05-11T01:14:00.426553+0000","last_clean":"2024-05-11T01:14:00.426553+0000","last_became_active":"2024-05-11T01:13:59.433946+0000","last_became_peered":"2024-05-11T01:13:59.433946+0000","last_unstale":"2024-05-11T01:14:00.426553+0000","last_undegraded":"2024-05-11T01:14:00.426553+0000","last_fullsized":"2024-05-11T01:14:00.426553+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:07:36.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425174+0000","last_change":"2024-05-11T01:13:58.432458+0000","last_active":"2024-05-11T01:14:00.425174+0000","last_peered":"2024-05-11T01:14:00.425174+0000","last_clean":"2024-05-11T01:14:00.425174+0000","last_became_active":"2024-05-11T01:13:58.432149+0000","last_became_peered":"2024-05-11T01:13:58.432149+0000","last_unstale":"2024-05-11T01:14:00.425174+0000","last_undegraded":"2024-05-11T01:14:00.425174+0000","last_fullsized":"2024-05-11T01:14:00.425174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424224+0000","last_change":"2024-05-11T01:13:59.432345+0000","last_active":"2024-05-11T01:14:00.424224+0000","last_peered":"2024-05-11T01:14:00.424224+0000","last_clean":"2024-05-11T01:14:00.424224+0000","last_became_active":"2024-05-11T01:13:59.432058+0000","last_became_peered":"2024-05-11T01:13:59.432058+0000","last_unstale":"2024-05-11T01:14:00.424224+0000","last_undegraded":"2024-05-11T01:14:00.424224+0000","last_fullsized":"2024-05-11T01:14:00.424224+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:38:32.025100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425955+0000","last_change":"2024-05-11T01:13:58.433877+0000","last_active":"2024-05-11T01:14:00.425955+0000","last_peered":"2024-05-11T01:14:00.425955+0000","last_clean":"2024-05-11T01:14:00.425955+0000","last_became_active":"2024-05-11T01:13:58.433584+0000","last_became_peered":"2024-05-11T01:13:58.433584+0000","last_unstale":"2024-05-11T01:14:00.425955+0000","last_undegraded":"2024-05-11T01:14:00.425955+0000","last_fullsized":"2024-05-11T01:14:00.425955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:20.805314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424533+0000","last_change":"2024-05-11T01:13:59.434011+0000","last_active":"2024-05-11T01:14:00.424533+0000","last_peered":"2024-05-11T01:14:00.424533+0000","last_clean":"2024-05-11T01:14:00.424533+0000","last_became_active":"2024-05-11T01:13:59.432337+0000","last_became_peered":"2024-05-11T01:13:59.432337+0000","last_unstale":"2024-05-11T01:14:00.424533+0000","last_undegraded":"2024-05-11T01:14:00.424533+0000","last_fullsized":"2024-05-11T01:14:00.424533+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424127+0000","last_change":"2024-05-11T01:13:58.435211+0000","last_active":"2024-05-11T01:14:00.424127+0000","last_peered":"2024-05-11T01:14:00.424127+0000","last_clean":"2024-05-11T01:14:00.424127+0000","last_became_active":"2024-05-11T01:13:58.434956+0000","last_became_peered":"2024-05-11T01:13:58.434956+0000","last_unstale":"2024-05-11T01:14:00.424127+0000","last_undegraded":"2024-05-11T01:14:00.424127+0000","last_fullsized":"2024-05-11T01:14:00.424127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424617+0000","last_change":"2024-05-11T01:13:59.432319+0000","last_active":"2024-05-11T01:14:00.424617+0000","last_peered":"2024-05-11T01:14:00.424617+0000","last_clean":"2024-05-11T01:14:00.424617+0000","last_became_active":"2024-05-11T01:13:59.432012+0000","last_became_peered":"2024-05-11T01:13:59.432012+0000","last_unstale":"2024-05-11T01:14:00.424617+0000","last_undegraded":"2024-05-11T01:14:00.424617+0000","last_fullsized":"2024-05-11T01:14:00.424617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:21.807568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424991+0000","last_change":"2024-05-11T01:13:58.433752+0000","last_active":"2024-05-11T01:14:00.424991+0000","last_peered":"2024-05-11T01:14:00.424991+0000","last_clean":"2024-05-11T01:14:00.424991+0000","last_became_active":"2024-05-11T01:13:58.433411+0000","last_became_peered":"2024-05-11T01:13:58.433411+0000","last_unstale":"2024-05-11T01:14:00.424991+0000","last_undegraded":"2024-05-11T01:14:00.424991+0000","last_fullsized":"2024-05-11T01:14:00.424991+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.423892+0000","last_change":"2024-05-11T01:13:59.446971+0000","last_active":"2024-05-11T01:14:00.423892+0000","last_peered":"2024-05-11T01:14:00.423892+0000","last_clean":"2024-05-11T01:14:00.423892+0000","last_became_active":"2024-05-11T01:13:59.446670+0000","last_became_peered":"2024-05-11T01:13:59.446670+0000","last_unstale":"2024-05-11T01:14:00.423892+0000","last_undegraded":"2024-05-11T01:14:00.423892+0000","last_fullsized":"2024-05-11T01:14:00.423892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:45:46.843918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.421509+0000","last_change":"2024-05-11T01:13:58.435876+0000","last_active":"2024-05-11T01:14:00.421509+0000","last_peered":"2024-05-11T01:14:00.421509+0000","last_clean":"2024-05-11T01:14:00.421509+0000","last_became_active":"2024-05-11T01:13:58.434062+0000","last_became_peered":"2024-05-11T01:13:58.434062+0000","last_unstale":"2024-05-11T01:14:00.421509+0000","last_undegraded":"2024-05-11T01:14:00.421509+0000","last_fullsized":"2024-05-11T01:14:00.421509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:12:26.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.422491+0000","last_change":"2024-05-11T01:13:59.445426+0000","last_active":"2024-05-11T01:14:00.422491+0000","last_peered":"2024-05-11T01:14:00.422491+0000","last_clean":"2024-05-11T01:14:00.422491+0000","last_became_active":"2024-05-11T01:13:59.445146+0000","last_became_peered":"2024-05-11T01:13:59.445146+0000","last_unstale":"2024-05-11T01:14:00.422491+0000","last_undegraded":"2024-05-11T01:14:00.422491+0000","last_fullsized":"2024-05-11T01:14:00.422491+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:57:36.879340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.424737+0000","last_change":"2024-05-11T01:13:58.434600+0000","last_active":"2024-05-11T01:14:00.424737+0000","last_peered":"2024-05-11T01:14:00.424737+0000","last_clean":"2024-05-11T01:14:00.424737+0000","last_became_active":"2024-05-11T01:13:58.434320+0000","last_became_peered":"2024-05-11T01:13:58.434320+0000","last_unstale":"2024-05-11T01:14:00.424737+0000","last_undegraded":"2024-05-11T01:14:00.424737+0000","last_fullsized":"2024-05-11T01:14:00.424737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"51'86","reported_seq":241,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-11T01:14:00.425450+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:14:00.425450+0000","last_peered":"2024-05-11T01:14:00.425450+0000","last_clean":"2024-05-11T01:14:00.425450+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:14:00.425450+0000","last_undegraded":"2024-05-11T01:14:00.425450+0000","last_fullsized":"2024-05-11T01:14:00.425450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":936289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542305,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":733184,"data_stored":346503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575007,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":729088,"data_stored":345901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575007,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54112,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317728,"statfs":{"total":96636764160,"available":96581353472,"internally_reserved":0,"allocated":737280,"data_stored":347443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44384,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327456,"statfs":{"total":96636764160,"available":96591314944,"internally_reserved":0,"allocated":737280,"data_stored":346469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":347465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58268,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313572,"statfs":{"total":96636764160,"available":96577097728,"internally_reserved":0,"allocated":729088,"data_stored":345901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36640,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335200,"statfs":{"total":96636764160,"available":96599244800,"internally_reserved":0,"allocated":1323008,"data_stored":936233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:14:03.848 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T01:14:03.848 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-11T01:14:03.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.997+0000 7f331f693640 1 Processor -- start 2024-05-11T01:14:03.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.997+0000 7f331f693640 1 -- start start 2024-05-11T01:14:03.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331f693640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f331815f430 0x7f33180b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331f693640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f33180d0700 0x7f33180d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:03.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331f693640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33180cfc60 con 0x7f33180b5420 2024-05-11T01:14:03.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331f693640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f33180d2a80 con 0x7f331815f430 2024-05-11T01:14:03.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331f693640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f3318163120 con 0x7f33180d0700 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331d42f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f331815f430 0x7f33180b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331d42f640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f331815f430 0x7f33180b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49572/0 (socket says 172.21.15.52:49572) 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331d42f640 1 -- 172.21.15.52:0/3341710518 learned_addr learned my addr 172.21.15.52:0/3341710518 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331d42f640 1 -- 172.21.15.52:0/3341710518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f33180d0700 msgr2=0x7f33180d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331dc30640 1 --2- 172.21.15.52:0/3341710518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f33180d0700 0x7f33180d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331d42f640 1 --2- 172.21.15.52:0/3341710518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f33180d0700 0x7f33180d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331d42f640 1 -- 172.21.15.52:0/3341710518 >> v1:172.21.15.119:6789/0 conn(0x7f33180b5420 legacy=0x7f33180d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.998+0000 7f331d42f640 1 -- 172.21.15.52:0/3341710518 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33180d24c0 con 0x7f331815f430 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.999+0000 7f331d42f640 1 --2- 172.21.15.52:0/3341710518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331815f430 0x7f33180b4ee0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3314009870 tx=0x7f331402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91638837f57eb7c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.999+0000 7f330ffff640 1 -- 172.21.15.52:0/3341710518 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314037070 con 0x7f331815f430 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.999+0000 7f330ffff640 1 -- 172.21.15.52:0/3341710518 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314032440 con 0x7f331815f430 2024-05-11T01:14:03.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:03.999+0000 7f330ffff640 1 -- 172.21.15.52:0/3341710518 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314032810 con 0x7f331815f430 2024-05-11T01:14:04.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.000+0000 7f331f693640 1 -- 172.21.15.52:0/3341710518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331815f430 msgr2=0x7f33180b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.000+0000 7f331f693640 1 --2- 172.21.15.52:0/3341710518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331815f430 0x7f33180b4ee0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3314009870 tx=0x7f331402a1c0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.000+0000 7f331f693640 1 -- 172.21.15.52:0/3341710518 shutdown_connections 2024-05-11T01:14:04.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.000+0000 7f331f693640 1 --2- 172.21.15.52:0/3341710518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f33180d0700 0x7f33180d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.000+0000 7f331f693640 1 --2- 172.21.15.52:0/3341710518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331815f430 0x7f33180b4ee0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.000+0000 7f331f693640 1 -- 172.21.15.52:0/3341710518 >> 172.21.15.52:0/3341710518 conn(0x7f331808ebb0 msgr2=0x7f331809e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:04.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.000+0000 7f331f693640 1 -- 172.21.15.52:0/3341710518 shutdown_connections 2024-05-11T01:14:04.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.000+0000 7f331f693640 1 -- 172.21.15.52:0/3341710518 wait complete. 2024-05-11T01:14:04.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.001+0000 7f331f693640 1 Processor -- start 2024-05-11T01:14:04.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.001+0000 7f331f693640 1 -- start start 2024-05-11T01:14:04.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331f693640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33180b5420 0x7f33180babb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331f693640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33180d0700 0x7f33180ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331f693640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f331815f430 0x7f331815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331f693640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33180d01e0 con 0x7f33180d0700 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331f693640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f33180cff00 con 0x7f33180b5420 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331f693640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f33180d2080 con 0x7f331815f430 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331cc2e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33180d0700 0x7f33180ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331cc2e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33180d0700 0x7f33180ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49582/0 (socket says 172.21.15.52:49582) 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331cc2e640 1 -- 172.21.15.52:0/792647847 learned_addr learned my addr 172.21.15.52:0/792647847 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331dc30640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f331815f430 0x7f331815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331d42f640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33180b5420 0x7f33180babb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331cc2e640 1 -- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f331815f430 msgr2=0x7f331815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331cc2e640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f331815f430 0x7f331815b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.002+0000 7f331cc2e640 1 -- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33180b5420 msgr2=0x7f33180babb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.003+0000 7f331cc2e640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33180b5420 0x7f33180babb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.003+0000 7f331cc2e640 1 -- 172.21.15.52:0/792647847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331815fd80 con 0x7f33180d0700 2024-05-11T01:14:04.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.003+0000 7f331d42f640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33180b5420 0x7f33180babb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:14:04.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.003+0000 7f331cc2e640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33180d0700 0x7f33180ccc90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f330800c430 tx=0x7f330800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:04.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.003+0000 7f330dffb640 1 -- 172.21.15.52:0/792647847 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308011790 con 0x7f33180d0700 2024-05-11T01:14:04.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.003+0000 7f330dffb640 1 -- 172.21.15.52:0/792647847 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3308017070 con 0x7f33180d0700 2024-05-11T01:14:04.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.003+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3318160060 con 0x7f33180d0700 2024-05-11T01:14:04.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.003+0000 7f330dffb640 1 -- 172.21.15.52:0/792647847 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308011bb0 con 0x7f33180d0700 2024-05-11T01:14:04.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.003+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3318160620 con 0x7f33180d0700 2024-05-11T01:14:04.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.004+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33180695d0 con 0x7f33180d0700 2024-05-11T01:14:04.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.005+0000 7f330dffb640 1 -- 172.21.15.52:0/792647847 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f330801c460 con 0x7f33180d0700 2024-05-11T01:14:04.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.005+0000 7f330dffb640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32fc076300 0x7f32fc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.005+0000 7f330dffb640 1 -- 172.21.15.52:0/792647847 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f33080976b0 con 0x7f33180d0700 2024-05-11T01:14:04.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.005+0000 7f331d42f640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32fc076300 0x7f32fc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.006+0000 7f331d42f640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32fc076300 0x7f32fc0787c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3314034000 tx=0x7f331402e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:04.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.010+0000 7f330dffb640 1 -- 172.21.15.52:0/792647847 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3308060de0 con 0x7f33180d0700 2024-05-11T01:14:04.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.272+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f33180d1e60 con 0x7f33180d0700 2024-05-11T01:14:04.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.273+0000 7f330dffb640 1 -- 172.21.15.52:0/792647847 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f33080606e0 con 0x7f33180d0700 2024-05-11T01:14:04.273 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-05-11T01:14:04.273 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQAdwz5mUO0yKhAAfS2pcT6nSB+NrJccryMO8A== 2024-05-11T01:14:04.273 INFO:teuthology.orchestra.run.smithi052.stdout: caps mds = "allow" 2024-05-11T01:14:04.273 INFO:teuthology.orchestra.run.smithi052.stdout: caps mon = "allow r" 2024-05-11T01:14:04.273 INFO:teuthology.orchestra.run.smithi052.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-11T01:14:04.273 INFO:teuthology.orchestra.run.smithi052.stderr:updated caps for client.0 2024-05-11T01:14:04.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32fc076300 msgr2=0x7f32fc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32fc076300 0x7f32fc0787c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3314034000 tx=0x7f331402e000 comp rx=0 tx=0).stop 2024-05-11T01:14:04.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33180d0700 msgr2=0x7f33180ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33180d0700 0x7f33180ccc90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f330800c430 tx=0x7f330800c8f0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 shutdown_connections 2024-05-11T01:14:04.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f32fc076300 0x7f32fc0787c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f331815f430 0x7f331815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33180d0700 0x7f33180ccc90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 --2- 172.21.15.52:0/792647847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33180b5420 0x7f33180babb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 >> 172.21.15.52:0/792647847 conn(0x7f331808ebb0 msgr2=0x7f33180d4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:04.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 shutdown_connections 2024-05-11T01:14:04.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.276+0000 7f331f693640 1 -- 172.21.15.52:0/792647847 wait complete. 2024-05-11T01:14:04.286 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:14:04.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.390+0000 7f01d4cf1640 1 Processor -- start 2024-05-11T01:14:04.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.390+0000 7f01d4cf1640 1 -- start start 2024-05-11T01:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01d4cf1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d006fbc0 0x7f01d006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01d4cf1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01d006c9c0 0x7f01d015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01d4cf1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d00b6f90 con 0x7f01d0159980 2024-05-11T01:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01d4cf1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f01d0159d60 con 0x7f01d006c9c0 2024-05-11T01:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01d4cf1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f01d0163d30 con 0x7f01d006fbc0 2024-05-11T01:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01ce59c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f01d0159980 0x7f01d0162ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55246/0 (socket says 172.21.15.52:55246) 2024-05-11T01:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01ce59c640 1 -- 172.21.15.52:0/2346060632 learned_addr learned my addr 172.21.15.52:0/2346060632 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01ced9d640 1 --2- 172.21.15.52:0/2346060632 >> v2:172.21.15.119:3300/0 conn(0x7f01d006c9c0 0x7f01d015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01c7fff640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d006fbc0 0x7f01d006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377417639 0 0) 0x7f01d00b6f90 con 0x7f01d0159980 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.391+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01bc003520 con 0x7f01d0159980 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4045572106 0 0) 0x7f01bc003520 con 0x7f01d0159980 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d006fbc0 msgr2=0x7f01d006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d006fbc0 0x7f01d006c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 >> v2:172.21.15.119:3300/0 conn(0x7f01d006c9c0 msgr2=0x7f01d015f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 --2- 172.21.15.52:0/2346060632 >> v2:172.21.15.119:3300/0 conn(0x7f01d006c9c0 0x7f01d015f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d0163a50 con 0x7f01d0159980 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f01c00016f0 con 0x7f01d0159980 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01ced9d640 1 --2- 172.21.15.52:0/2346060632 >> v2:172.21.15.119:3300/0 conn(0x7f01d006c9c0 0x7f01d015f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 >> v1:172.21.15.52:6789/0 conn(0x7f01d0159980 legacy=0x7f01d0162ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc003a00 0x7f01bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01bc004800 0x7f01bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d006fbc0 0x7f01bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0159d60 con 0x7f01bc004800 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0163d30 con 0x7f01bc003a00 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc00b9c0 con 0x7f01d006fbc0 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01c7fff640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01bc004800 0x7f01bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01ce59c640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc003a00 0x7f01bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01ced9d640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d006fbc0 0x7f01bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01c7fff640 1 -- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d006fbc0 msgr2=0x7f01bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01c7fff640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d006fbc0 0x7f01bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01c7fff640 1 -- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc003a00 msgr2=0x7f01bc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01c7fff640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc003a00 0x7f01bc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.392+0000 7f01c7fff640 1 -- 172.21.15.52:0/2346060632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc003520 con 0x7f01bc004800 2024-05-11T01:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.393+0000 7f01c7fff640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01bc004800 0x7f01bc004c80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009ae0 tx=0x7f01bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97349a17a010f7ef server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.393+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8036070 con 0x7f01bc004800 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.393+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b8005990 con 0x7f01bc004800 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.393+0000 7f01cdd9b640 1 -- 172.21.15.52:0/2346060632 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8009680 con 0x7f01bc004800 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.393+0000 7f01d4cf1640 1 -- 172.21.15.52:0/2346060632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01bc004800 msgr2=0x7f01bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.393+0000 7f01d4cf1640 1 --2- 172.21.15.52:0/2346060632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01bc004800 0x7f01bc004c80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009ae0 tx=0x7f01bc005830 comp rx=0 tx=0).stop 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.393+0000 7f01ced9d640 1 -- 172.21.15.52:0/2346060632 reap_dead start 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.394+0000 7f01d4cf1640 1 -- 172.21.15.52:0/2346060632 shutdown_connections 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.394+0000 7f01d4cf1640 1 -- 172.21.15.52:0/2346060632 >> 172.21.15.52:0/2346060632 conn(0x7f01d00b3af0 msgr2=0x7f01d00b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.394+0000 7f01d4cf1640 1 -- 172.21.15.52:0/2346060632 shutdown_connections 2024-05-11T01:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.394+0000 7f01d4cf1640 1 -- 172.21.15.52:0/2346060632 wait complete. 2024-05-11T01:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.395+0000 7f01d4cf1640 1 Processor -- start 2024-05-11T01:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.395+0000 7f01d4cf1640 1 -- start start 2024-05-11T01:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01d4cf1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc004800 0x7f01d0146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01d4cf1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d01531d0 0x7f01d00b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01d4cf1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d00b0780 0x7f01d01512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01d4cf1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f01d006f6a0 con 0x7f01d00b0780 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01d4cf1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f01d006f3c0 con 0x7f01bc004800 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01d4cf1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f01d01635f0 con 0x7f01d01531d0 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ced9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d00b0780 0x7f01d01512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ced9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d00b0780 0x7f01d01512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49594/0 (socket says 172.21.15.52:49594) 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ced9d640 1 -- 172.21.15.52:0/4257852561 learned_addr learned my addr 172.21.15.52:0/4257852561 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ce59c640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc004800 0x7f01d0146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01c7fff640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d01531d0 0x7f01d00b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ced9d640 1 -- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d01531d0 msgr2=0x7f01d00b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ced9d640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d01531d0 0x7f01d00b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ced9d640 1 -- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc004800 msgr2=0x7f01d0146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ced9d640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc004800 0x7f01d0146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ced9d640 1 -- 172.21.15.52:0/4257852561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d015bd70 con 0x7f01d00b0780 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ce59c640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc004800 0x7f01d0146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:14:04.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01ced9d640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d00b0780 0x7f01d01512a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f01d006d4e0 tx=0x7f01c800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01c6ffd640 1 -- 172.21.15.52:0/4257852561 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8003990 con 0x7f01d00b0780 2024-05-11T01:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01c6ffd640 1 -- 172.21.15.52:0/4257852561 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c8003b30 con 0x7f01d00b0780 2024-05-11T01:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01c6ffd640 1 -- 172.21.15.52:0/4257852561 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8004020 con 0x7f01d00b0780 2024-05-11T01:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.396+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d015acd0 con 0x7f01d00b0780 2024-05-11T01:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.397+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d015b290 con 0x7f01d00b0780 2024-05-11T01:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.397+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d0150eb0 con 0x7f01d00b0780 2024-05-11T01:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.398+0000 7f01c6ffd640 1 -- 172.21.15.52:0/4257852561 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f01c801f020 con 0x7f01d00b0780 2024-05-11T01:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.398+0000 7f01c6ffd640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f01b0076300 0x7f01b00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:04.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.399+0000 7f01c6ffd640 1 -- 172.21.15.52:0/4257852561 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f01c8097d30 con 0x7f01d00b0780 2024-05-11T01:14:04.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.399+0000 7f01ce59c640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f01b0076300 0x7f01b00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:04.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.399+0000 7f01ce59c640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f01b0076300 0x7f01b00787c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01c0001010 tx=0x7f01c0009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:04.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.403+0000 7f01c6ffd640 1 -- 172.21.15.52:0/4257852561 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f01c8010040 con 0x7f01d00b0780 2024-05-11T01:14:04.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.633+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01d0069780 con 0x7f01d00b0780 2024-05-11T01:14:04.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.633+0000 7f01c6ffd640 1 -- 172.21.15.52:0/4257852561 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5178 (secure 0 0 0) 0x7f01c8061460 con 0x7f01d00b0780 2024-05-11T01:14:04.633 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:14:04.633 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/4021589378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4021589378},{"type":"v1","addr":"172.21.15.119:6836","nonce":4021589378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/456790373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":456790373},{"type":"v1","addr":"172.21.15.119:6837","nonce":456790373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/1460397063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1460397063},{"type":"v1","addr":"172.21.15.52:6834","nonce":1460397063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:13:58.799356+0000","modified":"2024-05-11T01:14:01.810259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/59064433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":59064433},{"type":"v1","addr":"172.21.15.52:6835","nonce":59064433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5649,"qdb_cluster":[5649]},"id":3}]} 2024-05-11T01:14:04.633 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-05-11T01:14:04.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f01b0076300 msgr2=0x7f01b00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f01b0076300 0x7f01b00787c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01c0001010 tx=0x7f01c0009040 comp rx=0 tx=0).stop 2024-05-11T01:14:04.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d00b0780 msgr2=0x7f01d01512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:04.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d00b0780 0x7f01d01512a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f01d006d4e0 tx=0x7f01c800f580 comp rx=0 tx=0).stop 2024-05-11T01:14:04.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 shutdown_connections 2024-05-11T01:14:04.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f01b0076300 0x7f01b00787c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d00b0780 0x7f01d01512a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f01d01531d0 0x7f01d00b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 --2- 172.21.15.52:0/4257852561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01bc004800 0x7f01d0146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:04.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 >> 172.21.15.52:0/4257852561 conn(0x7f01d00b3af0 msgr2=0x7f01d00b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:04.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 shutdown_connections 2024-05-11T01:14:04.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:04.635+0000 7f01d4cf1640 1 -- 172.21.15.52:0/4257852561 wait complete. 2024-05-11T01:14:05.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T01:13:58.799356+0000', 'modified': '2024-05-11T01:14:01.810259+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5649}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5649': {'gid': 5649, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6835/59064433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6833', 'nonce': 59064433}, {'type': 'v1', 'addr': '172.21.15.52:6835', 'nonce': 59064433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5649, 'qdb_cluster': [5649]} 2024-05-11T01:14:05.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T01:14:05.048 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-11T01:14:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.159+0000 7f25283b6640 1 Processor -- start 2024-05-11T01:14:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.159+0000 7f25283b6640 1 -- start start 2024-05-11T01:14:05.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f25283b6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2520160480 0x7f25200d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f25283b6640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f25200d32c0 0x7f25200d36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f25283b6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2520163250 con 0x7f252015f430 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f25283b6640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f2520163530 con 0x7f25200d32c0 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f25283b6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f2520163810 con 0x7f2520160480 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f2526953640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f25200d32c0 0x7f25200d36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f2526953640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f25200d32c0 0x7f25200d36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49600/0 (socket says 172.21.15.52:49600) 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f2526953640 1 -- 172.21.15.52:0/1238136364 learned_addr learned my addr 172.21.15.52:0/1238136364 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f2525951640 1 --2- 172.21.15.52:0/1238136364 >> v2:172.21.15.119:3300/0 conn(0x7f2520160480 0x7f25200d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f2526953640 1 -- 172.21.15.52:0/1238136364 >> v2:172.21.15.119:3300/0 conn(0x7f2520160480 msgr2=0x7f25200d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f2526953640 1 --2- 172.21.15.52:0/1238136364 >> v2:172.21.15.119:3300/0 conn(0x7f2520160480 0x7f25200d2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f2526953640 1 -- 172.21.15.52:0/1238136364 >> v1:172.21.15.119:6789/0 conn(0x7f252015f430 legacy=0x7f252015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f2526953640 1 -- 172.21.15.52:0/1238136364 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2520163af0 con 0x7f25200d32c0 2024-05-11T01:14:05.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.160+0000 7f2525951640 1 --2- 172.21.15.52:0/1238136364 >> v2:172.21.15.119:3300/0 conn(0x7f2520160480 0x7f25200d2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:14:05.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.161+0000 7f2526953640 1 --2- 172.21.15.52:0/1238136364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 0x7f25200d36c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f25200b4840 tx=0x7f251002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e5936210cb7b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.161+0000 7f2525150640 1 -- 172.21.15.52:0/1238136364 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510037070 con 0x7f25200d32c0 2024-05-11T01:14:05.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.161+0000 7f2525150640 1 -- 172.21.15.52:0/1238136364 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2510032440 con 0x7f25200d32c0 2024-05-11T01:14:05.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.161+0000 7f2525150640 1 -- 172.21.15.52:0/1238136364 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510032810 con 0x7f25200d32c0 2024-05-11T01:14:05.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.162+0000 7f25283b6640 1 -- 172.21.15.52:0/1238136364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 msgr2=0x7f25200d36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.162+0000 7f25283b6640 1 --2- 172.21.15.52:0/1238136364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 0x7f25200d36c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f25200b4840 tx=0x7f251002a1e0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.162+0000 7f25283b6640 1 -- 172.21.15.52:0/1238136364 shutdown_connections 2024-05-11T01:14:05.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.162+0000 7f25283b6640 1 --2- 172.21.15.52:0/1238136364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 0x7f25200d36c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.162+0000 7f25283b6640 1 --2- 172.21.15.52:0/1238136364 >> v2:172.21.15.119:3300/0 conn(0x7f2520160480 0x7f25200d2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.162+0000 7f25283b6640 1 -- 172.21.15.52:0/1238136364 >> 172.21.15.52:0/1238136364 conn(0x7f252008eb70 msgr2=0x7f252009e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:05.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.162+0000 7f25283b6640 1 -- 172.21.15.52:0/1238136364 shutdown_connections 2024-05-11T01:14:05.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.162+0000 7f25283b6640 1 -- 172.21.15.52:0/1238136364 wait complete. 2024-05-11T01:14:05.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.163+0000 7f25283b6640 1 Processor -- start 2024-05-11T01:14:05.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.163+0000 7f25283b6640 1 -- start start 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f25283b6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 0x7f252015a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f25283b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252015f430 0x7f25201f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f25283b6640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2520160480 0x7f25200ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f25283b6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2520163030 con 0x7f25200d32c0 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f25283b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2520162eb0 con 0x7f252015f430 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f25283b6640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f25201608f0 con 0x7f2520160480 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526152640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 0x7f252015a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526152640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 0x7f252015a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49610/0 (socket says 172.21.15.52:49610) 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526152640 1 -- 172.21.15.52:0/3619699046 learned_addr learned my addr 172.21.15.52:0/3619699046 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2525951640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252015f430 0x7f25201f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526953640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2520160480 0x7f25200ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526152640 1 -- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2520160480 msgr2=0x7f25200ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526152640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2520160480 0x7f25200ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526152640 1 -- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252015f430 msgr2=0x7f25201f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526152640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252015f430 0x7f25201f5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526152640 1 -- 172.21.15.52:0/3619699046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25201f6510 con 0x7f25200d32c0 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2525951640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252015f430 0x7f25201f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f2526152640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 0x7f252015a6a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f251800c870 tx=0x7f251800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f251effd640 1 -- 172.21.15.52:0/3619699046 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518003b70 con 0x7f25200d32c0 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f251effd640 1 -- 172.21.15.52:0/3619699046 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2518003d10 con 0x7f25200d32c0 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f25283b6640 1 -- 172.21.15.52:0/3619699046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25201f52a0 con 0x7f25200d32c0 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f251effd640 1 -- 172.21.15.52:0/3619699046 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251800aa90 con 0x7f25200d32c0 2024-05-11T01:14:05.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.164+0000 7f25283b6640 1 -- 172.21.15.52:0/3619699046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25201f5860 con 0x7f25200d32c0 2024-05-11T01:14:05.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.166+0000 7f251effd640 1 -- 172.21.15.52:0/3619699046 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f251800ac30 con 0x7f25200d32c0 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.167+0000 7f251effd640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f250c0762d0 0x7f250c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.167+0000 7f251effd640 1 -- 172.21.15.52:0/3619699046 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f2518097770 con 0x7f25200d32c0 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.167+0000 7f2525951640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f250c0762d0 0x7f250c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.168+0000 7f252b1ad740 1 Processor -- start 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.168+0000 7f2525951640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f250c0762d0 0x7f250c078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25200cbc90 tx=0x7f2514008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.168+0000 7f252b1ad740 1 -- start start 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.168+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826722800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.168+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b8267203d0 0x55b826722d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.168+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8266f2c50 0x55b8266f6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.168+0000 7f252b1ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b82665c8d0 con 0x55b82671fff0 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.168+0000 7f252b1ad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b826615c10 con 0x55b8266f2c50 2024-05-11T01:14:05.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.168+0000 7f252b1ad740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x55b826658890 con 0x55b8267203d0 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526152640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826722800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526152640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826722800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49626/0 (socket says 172.21.15.52:49626) 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526152640 1 -- 172.21.15.52:0/1959880646 learned_addr learned my addr 172.21.15.52:0/1959880646 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526953640 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8266f2c50 0x55b8266f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2525951640 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b8267203d0 0x55b826722d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526152640 1 -- 172.21.15.52:0/1959880646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b8267203d0 msgr2=0x55b826722d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526152640 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b8267203d0 0x55b826722d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526152640 1 -- 172.21.15.52:0/1959880646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8266f2c50 msgr2=0x55b8266f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526152640 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8266f2c50 0x55b8266f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526152640 1 -- 172.21.15.52:0/1959880646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82661d5d0 con 0x55b82671fff0 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526953640 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8266f2c50 0x55b8266f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2526152640 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826722800 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f251805c820 tx=0x7f25180606a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62f34bf31a6ba75 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2507fff640 1 -- 172.21.15.52:0/1959880646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518060ea0 con 0x55b82671fff0 2024-05-11T01:14:05.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2507fff640 1 -- 172.21.15.52:0/1959880646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25180607a0 con 0x55b82671fff0 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.169+0000 7f2507fff640 1 -- 172.21.15.52:0/1959880646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518075d60 con 0x55b82671fff0 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.170+0000 7f252b1ad740 1 -- 172.21.15.52:0/1959880646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 msgr2=0x55b826722800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.170+0000 7f252b1ad740 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826722800 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f251805c820 tx=0x7f25180606a0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.170+0000 7f252b1ad740 1 -- 172.21.15.52:0/1959880646 shutdown_connections 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.170+0000 7f252b1ad740 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8266f2c50 0x55b8266f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.170+0000 7f252b1ad740 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b8267203d0 0x55b826722d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.170+0000 7f252b1ad740 1 --2- 172.21.15.52:0/1959880646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826722800 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.170+0000 7f252b1ad740 1 -- 172.21.15.52:0/1959880646 >> 172.21.15.52:0/1959880646 conn(0x55b8266f4ec0 msgr2=0x55b82673d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.170+0000 7f252b1ad740 1 -- 172.21.15.52:0/1959880646 shutdown_connections 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.170+0000 7f252b1ad740 1 -- 172.21.15.52:0/1959880646 wait complete. 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.171+0000 7f252b1ad740 1 -- start start 2024-05-11T01:14:05.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.171+0000 7f252b1ad740 1 Processor -- start 2024-05-11T01:14:05.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.171+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 0x55b82673ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b8267416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b826742f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f252b1ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b8266f8070 con 0x55b8267203d0 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f252b1ad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b826619760 con 0x55b82671fff0 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f252b1ad740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x55b8266f8350 con 0x55b826742aa0 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f252b1ad740 10 client.0 _ll_register_callbacks cb 0x55b826652cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2525951640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b8267416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2525951640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b8267416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49636/0 (socket says 172.21.15.52:49636) 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2525951640 1 -- 172.21.15.52:0/872233458 learned_addr learned my addr 172.21.15.52:0/872233458 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2526152640 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 0x55b82673ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2526953640 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b826742f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2525951640 1 -- 172.21.15.52:0/872233458 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 msgr2=0x55b826742f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2525951640 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b826742f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2525951640 1 -- 172.21.15.52:0/872233458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 msgr2=0x55b82673ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2525951640 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 0x55b82673ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2525951640 1 -- 172.21.15.52:0/872233458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82636e2d0 con 0x55b8267203d0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f2525951640 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b8267416c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f251400fff0 tx=0x7f2514004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda0d6a03e554da9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f25057fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f25057fa640 1 -- 172.21.15.52:0/872233458 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514015820 con 0x55b8267203d0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f25057fa640 1 -- 172.21.15.52:0/872233458 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f25140159c0 con 0x55b8267203d0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.172+0000 7f252b1ad740 1 -- 172.21.15.52:0/872233458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25180607a0 con 0x55b8267203d0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f25057fa640 1 -- 172.21.15.52:0/872233458 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f251402e050 con 0x55b8267203d0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f252b1ad740 10 client.5784 fetch_fsmap learned FSMap version 24 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f252b1ad740 1 -- 172.21.15.52:0/872233458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b8264386b0 con 0x55b8267203d0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f25057fa640 1 -- 172.21.15.52:0/872233458 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25180607a0 con 0x55b8267203d0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f25057fa640 1 -- 172.21.15.52:0/872233458 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f2514031020 con 0x55b8267203d0 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f252b1ad740 10 client.5784 fetch_fsmap finished waiting for FSMap version 24 2024-05-11T01:14:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f252b1ad740 10 client.5784 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f252b1ad740 20 client.5784 populate_metadata read hostname 'smithi052' 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f252b1ad740 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] conn(0x55b826723ee0 0x55b82675c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f252b1ad740 4 client.5784 mds_command: new command op to 5649 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f252b1ad740 1 -- 172.21.15.52:0/872233458 --> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8265e0760 con 0x55b826723ee0 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f2526152640 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] conn(0x55b826723ee0 0x55b82675c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f25057fa640 10 client.5784 ms_handle_connect on v2:172.21.15.52:6833/59064433 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.173+0000 7f2526152640 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] conn(0x55b826723ee0 0x55b82675c910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d235ae563f9d34 server_cookie=6f6fe66579404464 in_seq=0 out_seq=0 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f25057fa640 1 -- 172.21.15.52:0/872233458 <== mds.0 v2:172.21.15.52:6833/59064433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b8265e0760 con 0x55b826723ee0 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f25057fa640 10 client.5784 handle_command_reply: tid=0 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 client.5784 shutdown 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 -- 172.21.15.52:0/872233458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 msgr2=0x55b8267416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b8267416c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f251400fff0 tx=0x7f2514004730 comp rx=0 tx=0).stop 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 -- 172.21.15.52:0/872233458 shutdown_connections 2024-05-11T01:14:05.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] conn(0x55b826723ee0 0x55b82675c910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b826742f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f25057fa640 0 client.5784 ms_handle_reset on v2:172.21.15.52:6833/59064433 2024-05-11T01:14:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b8267416c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 --2- 172.21.15.52:0/872233458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 0x55b82673ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 -- 172.21.15.52:0/872233458 >> 172.21.15.52:0/872233458 conn(0x55b8266f2c50 msgr2=0x55b8266f6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 -- 172.21.15.52:0/872233458 shutdown_connections 2024-05-11T01:14:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.174+0000 7f252b1ad740 1 -- 172.21.15.52:0/872233458 wait complete. 2024-05-11T01:14:05.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.175+0000 7f252b1ad740 20 client.5784 trim_cache size 0 max 16384 2024-05-11T01:14:05.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.193+0000 7f252b1ad740 1 Processor -- start 2024-05-11T01:14:05.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.193+0000 7f252b1ad740 1 -- start start 2024-05-11T01:14:05.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826742ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8267203d0 0x55b826743400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b82673d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f252b1ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2520200f70 con 0x55b82671fff0 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f252b1ad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f25200ccf00 con 0x55b8267203d0 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f252b1ad740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f2520200920 con 0x55b826742aa0 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f2526152640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826742ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f2526152640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826742ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49640/0 (socket says 172.21.15.52:49640) 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f2526152640 1 -- 172.21.15.52:0/2874153488 learned_addr learned my addr 172.21.15.52:0/2874153488 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f2526152640 1 -- 172.21.15.52:0/2874153488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 msgr2=0x55b82673d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f2526152640 1 --2- 172.21.15.52:0/2874153488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b82673d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f2526152640 1 -- 172.21.15.52:0/2874153488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8267203d0 msgr2=0x55b826743400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f2526152640 1 --2- 172.21.15.52:0/2874153488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8267203d0 0x55b826743400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f2526152640 1 -- 172.21.15.52:0/2874153488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82671e180 con 0x55b82671fff0 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f2526152640 1 --2- 172.21.15.52:0/2874153488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826742ec0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2518060730 tx=0x7f2518066000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc854a7bf3df4f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.194+0000 7f25077fe640 1 -- 172.21.15.52:0/2874153488 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25180607a0 con 0x55b82671fff0 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f25077fe640 1 -- 172.21.15.52:0/2874153488 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2518064b50 con 0x55b82671fff0 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f25077fe640 1 -- 172.21.15.52:0/2874153488 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518075da0 con 0x55b82671fff0 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f252b1ad740 1 -- 172.21.15.52:0/2874153488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 msgr2=0x55b826742ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f252b1ad740 1 --2- 172.21.15.52:0/2874153488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826742ec0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2518060730 tx=0x7f2518066000 comp rx=0 tx=0).stop 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f252b1ad740 1 -- 172.21.15.52:0/2874153488 shutdown_connections 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f252b1ad740 1 --2- 172.21.15.52:0/2874153488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b82673d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f252b1ad740 1 --2- 172.21.15.52:0/2874153488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8267203d0 0x55b826743400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f252b1ad740 1 --2- 172.21.15.52:0/2874153488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82671fff0 0x55b826742ec0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f252b1ad740 1 -- 172.21.15.52:0/2874153488 >> 172.21.15.52:0/2874153488 conn(0x55b8266f2c50 msgr2=0x55b8266f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f252b1ad740 1 -- 172.21.15.52:0/2874153488 shutdown_connections 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.195+0000 7f252b1ad740 1 -- 172.21.15.52:0/2874153488 wait complete. 2024-05-11T01:14:05.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.196+0000 7f252b1ad740 1 -- start start 2024-05-11T01:14:05.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.196+0000 7f252b1ad740 1 Processor -- start 2024-05-11T01:14:05.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 0x55b826742ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b826743400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f252b1ad740 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b82673d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f252b1ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b82673f810 con 0x55b8267203d0 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f252b1ad740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b826741c00 con 0x55b82671fff0 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f252b1ad740 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x55b826725ca0 con 0x55b826742aa0 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f252b1ad740 10 client.0 _ll_register_callbacks cb 0x7f25140041e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2525951640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b826743400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2525951640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b826743400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49656/0 (socket says 172.21.15.52:49656) 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2525951640 1 -- 172.21.15.52:0/8887872 learned_addr learned my addr 172.21.15.52:0/8887872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2526152640 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 0x55b826742ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2526953640 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b82673d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2525951640 1 -- 172.21.15.52:0/8887872 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 msgr2=0x55b82673d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2525951640 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b82673d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2525951640 1 -- 172.21.15.52:0/8887872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 msgr2=0x55b826742ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2525951640 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 0x55b826742ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2525951640 1 -- 172.21.15.52:0/8887872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82673a400 con 0x55b8267203d0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2526953640 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b82673d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f2525951640 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b826743400 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2514000c00 tx=0x7f25140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6851c6b800554c70 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f25057fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f25057fa640 1 -- 172.21.15.52:0/8887872 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514036020 con 0x55b8267203d0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f25057fa640 1 -- 172.21.15.52:0/8887872 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7f2514022860 con 0x55b8267203d0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.197+0000 7f252b1ad740 1 -- 172.21.15.52:0/8887872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2518064b50 con 0x55b8267203d0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f25057fa640 1 -- 172.21.15.52:0/8887872 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2514040050 con 0x55b8267203d0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f252b1ad740 10 client.5790 fetch_fsmap learned FSMap version 24 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f252b1ad740 1 -- 172.21.15.52:0/8887872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b8267bbde0 con 0x55b8267203d0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f25057fa640 1 -- 172.21.15.52:0/8887872 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2518064b50 con 0x55b8267203d0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f25057fa640 1 -- 172.21.15.52:0/8887872 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f2514043020 con 0x55b8267203d0 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f252b1ad740 10 client.5790 fetch_fsmap finished waiting for FSMap version 24 2024-05-11T01:14:05.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f252b1ad740 10 client.5790 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f252b1ad740 20 client.5790 populate_metadata read hostname 'smithi052' 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f252b1ad740 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] conn(0x55b8267364a0 0x55b826736880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f252b1ad740 4 client.5790 mds_command: new command op to 5649 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f252b1ad740 1 -- 172.21.15.52:0/8887872 --> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] -- command(tid 0: {"prefix": "status"}) -- 0x55b82666a4e0 con 0x55b8267364a0 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f2526152640 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] conn(0x55b8267364a0 0x55b826736880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f25057fa640 10 client.5790 ms_handle_connect on v2:172.21.15.52:6833/59064433 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.198+0000 7f2526152640 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] conn(0x55b8267364a0 0x55b826736880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35eb2c816dc1dc8a server_cookie=ed3403662ec9f2ee in_seq=0 out_seq=0 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f25057fa640 1 -- 172.21.15.52:0/8887872 <== mds.0 v2:172.21.15.52:6833/59064433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b82666a4e0 con 0x55b8267364a0 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f25057fa640 10 client.5790 handle_command_reply: tid=0 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 client.5790 shutdown 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 -- 172.21.15.52:0/8887872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 msgr2=0x55b826743400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b826743400 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2514000c00 tx=0x7f25140046c0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 -- 172.21.15.52:0/8887872 shutdown_connections 2024-05-11T01:14:05.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.52:6833/59064433,v1:172.21.15.52:6835/59064433] conn(0x55b8267364a0 0x55b826736880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55b826742aa0 0x55b82673d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8267203d0 0x55b826743400 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 --2- 172.21.15.52:0/8887872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b82671fff0 0x55b826742ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f25057fa640 0 client.5790 ms_handle_reset on v2:172.21.15.52:6833/59064433 2024-05-11T01:14:05.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 -- 172.21.15.52:0/8887872 >> 172.21.15.52:0/8887872 conn(0x55b8266f2c50 msgr2=0x55b8267227c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:05.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 -- 172.21.15.52:0/8887872 shutdown_connections 2024-05-11T01:14:05.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 1 -- 172.21.15.52:0/8887872 wait complete. 2024-05-11T01:14:05.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f252b1ad740 20 client.5790 trim_cache size 0 max 16384 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "f8411cb1-0508-49cc-906f-2364b6835afe", 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 5649, 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 5.7809935640000001, 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 24, 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 61, 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 61, 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 11.089261207 2024-05-11T01:14:05.205 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:14:05.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f25283b6640 1 -- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f250c0762d0 msgr2=0x7f250c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f25283b6640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f250c0762d0 0x7f250c078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25200cbc90 tx=0x7f2514008040 comp rx=0 tx=0).stop 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f25283b6640 1 -- 172.21.15.52:0/3619699046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 msgr2=0x7f252015a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f25283b6640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 0x7f252015a6a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f251800c870 tx=0x7f251800cd30 comp rx=0 tx=0).stop 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.199+0000 7f25283b6640 1 -- 172.21.15.52:0/3619699046 shutdown_connections 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.200+0000 7f25283b6640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f250c0762d0 0x7f250c078790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.200+0000 7f25283b6640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2520160480 0x7f25200ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.200+0000 7f25283b6640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252015f430 0x7f25201f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.200+0000 7f25283b6640 1 --2- 172.21.15.52:0/3619699046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25200d32c0 0x7f252015a6a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.200+0000 7f25283b6640 1 -- 172.21.15.52:0/3619699046 >> 172.21.15.52:0/3619699046 conn(0x7f252008eb70 msgr2=0x7f25200d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.200+0000 7f25283b6640 1 -- 172.21.15.52:0/3619699046 shutdown_connections 2024-05-11T01:14:05.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.200+0000 7f25283b6640 1 -- 172.21.15.52:0/3619699046 wait complete. 2024-05-11T01:14:05.210 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T01:14:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.314+0000 7fd76ac09640 1 Processor -- start 2024-05-11T01:14:05.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.315+0000 7fd76ac09640 1 -- start start 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.315+0000 7fd76ac09640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd76415f4d0 0x7fd7640d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.315+0000 7fd76ac09640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7640d32c0 0x7fd7640ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.315+0000 7fd76ac09640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd76406e710 con 0x7fd76406dfa0 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.315+0000 7fd76ac09640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fd7640d2570 con 0x7fd76415f4d0 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.315+0000 7fd76ac09640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fd76415f8b0 con 0x7fd7640d32c0 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.315+0000 7fd7689a5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd76415f4d0 0x7fd7640d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd763fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fd76406dfa0 0x7fd7640d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55258/0 (socket says 172.21.15.52:55258) 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd7689a5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fd76415f4d0 0x7fd7640d3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49666/0 (socket says 172.21.15.52:49666) 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd763fff640 1 -- 172.21.15.52:0/3442324135 learned_addr learned my addr 172.21.15.52:0/3442324135 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd7691a6640 1 --2- 172.21.15.52:0/3442324135 >> v2:172.21.15.119:3300/0 conn(0x7fd7640d32c0 0x7fd7640ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd7689a5640 1 -- 172.21.15.52:0/3442324135 >> v2:172.21.15.119:3300/0 conn(0x7fd7640d32c0 msgr2=0x7fd7640ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:05.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd7689a5640 1 --2- 172.21.15.52:0/3442324135 >> v2:172.21.15.119:3300/0 conn(0x7fd7640d32c0 0x7fd7640ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd7689a5640 1 -- 172.21.15.52:0/3442324135 >> v1:172.21.15.52:6789/0 conn(0x7fd76406dfa0 legacy=0x7fd7640d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd7637fe640 1 -- 172.21.15.52:0/3442324135 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1312673712 0 0) 0x7fd76406e710 con 0x7fd76406dfa0 2024-05-11T01:14:05.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd7689a5640 1 -- 172.21.15.52:0/3442324135 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7640d1cd0 con 0x7fd76415f4d0 2024-05-11T01:14:05.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.316+0000 7fd7691a6640 1 --2- 172.21.15.52:0/3442324135 >> v2:172.21.15.119:3300/0 conn(0x7fd7640d32c0 0x7fd7640ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:14:05.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd7689a5640 1 --2- 172.21.15.52:0/3442324135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 0x7fd7640d3750 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd7640d0a40 tx=0x7fd75c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edfe3e61e5745dd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd7637fe640 1 -- 172.21.15.52:0/3442324135 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c037070 con 0x7fd76415f4d0 2024-05-11T01:14:05.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd7637fe640 1 -- 172.21.15.52:0/3442324135 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75c032440 con 0x7fd76415f4d0 2024-05-11T01:14:05.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd7637fe640 1 -- 172.21.15.52:0/3442324135 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c032810 con 0x7fd76415f4d0 2024-05-11T01:14:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd76ac09640 1 -- 172.21.15.52:0/3442324135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 msgr2=0x7fd7640d3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd76ac09640 1 --2- 172.21.15.52:0/3442324135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 0x7fd7640d3750 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd7640d0a40 tx=0x7fd75c02a270 comp rx=0 tx=0).stop 2024-05-11T01:14:05.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd76ac09640 1 -- 172.21.15.52:0/3442324135 shutdown_connections 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd76ac09640 1 --2- 172.21.15.52:0/3442324135 >> v2:172.21.15.119:3300/0 conn(0x7fd7640d32c0 0x7fd7640ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd76ac09640 1 --2- 172.21.15.52:0/3442324135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 0x7fd7640d3750 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd76ac09640 1 -- 172.21.15.52:0/3442324135 >> 172.21.15.52:0/3442324135 conn(0x7fd76409ba20 msgr2=0x7fd76409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd76ac09640 1 -- 172.21.15.52:0/3442324135 shutdown_connections 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.317+0000 7fd76ac09640 1 -- 172.21.15.52:0/3442324135 wait complete. 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.318+0000 7fd76ac09640 1 Processor -- start 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.319+0000 7fd76ac09640 1 -- start start 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.319+0000 7fd76ac09640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76406dfa0 0x7fd76414f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.319+0000 7fd76ac09640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7640d32c0 0x7fd7640714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.319+0000 7fd76ac09640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 0x7fd764071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.319+0000 7fd76ac09640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd76406e3e0 con 0x7fd76415f4d0 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.319+0000 7fd76ac09640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd7640d2290 con 0x7fd76406dfa0 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.319+0000 7fd76ac09640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fd7640cec10 con 0x7fd7640d32c0 2024-05-11T01:14:05.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7691a6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 0x7fd764071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7691a6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 0x7fd764071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49670/0 (socket says 172.21.15.52:49670) 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7691a6640 1 -- 172.21.15.52:0/3849008439 learned_addr learned my addr 172.21.15.52:0/3849008439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7689a5640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76406dfa0 0x7fd76414f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd763fff640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7640d32c0 0x7fd7640714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7691a6640 1 -- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7640d32c0 msgr2=0x7fd7640714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7691a6640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7640d32c0 0x7fd7640714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7691a6640 1 -- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76406dfa0 msgr2=0x7fd76414f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7691a6640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76406dfa0 0x7fd76414f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7691a6640 1 -- 172.21.15.52:0/3849008439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7640b6ae0 con 0x7fd76415f4d0 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7691a6640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 0x7fd764071a00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00e5e0 tx=0x7fd74c00eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7617fa640 1 -- 172.21.15.52:0/3849008439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c00f040 con 0x7fd76415f4d0 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7617fa640 1 -- 172.21.15.52:0/3849008439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74c004230 con 0x7fd76415f4d0 2024-05-11T01:14:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7640b6dc0 con 0x7fd76415f4d0 2024-05-11T01:14:05.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd7617fa640 1 -- 172.21.15.52:0/3849008439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c003610 con 0x7fd76415f4d0 2024-05-11T01:14:05.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.320+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7640b7380 con 0x7fd76415f4d0 2024-05-11T01:14:05.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.321+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7640cfcf0 con 0x7fd76415f4d0 2024-05-11T01:14:05.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.322+0000 7fd7617fa640 1 -- 172.21.15.52:0/3849008439 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fd74c0037b0 con 0x7fd76415f4d0 2024-05-11T01:14:05.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.323+0000 7fd7617fa640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fd7440762d0 0x7fd744078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:05.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.323+0000 7fd7617fa640 1 -- 172.21.15.52:0/3849008439 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7fd74c0974b0 con 0x7fd76415f4d0 2024-05-11T01:14:05.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.323+0000 7fd7689a5640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fd7440762d0 0x7fd744078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:05.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.323+0000 7fd7689a5640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fd7440762d0 0x7fd744078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd75c034000 tx=0x7fd75c02e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:05.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.327+0000 7fd7617fa640 1 -- 172.21.15.52:0/3849008439 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd7640cfcf0 con 0x7fd76415f4d0 2024-05-11T01:14:05.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.566+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd76415ecf0 con 0x7fd76415f4d0 2024-05-11T01:14:05.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.566+0000 7fd7617fa640 1 -- 172.21.15.52:0/3849008439 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7fd74c017020 con 0x7fd76415f4d0 2024-05-11T01:14:05.565 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T01:14:05.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fd7440762d0 msgr2=0x7fd744078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fd7440762d0 0x7fd744078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd75c034000 tx=0x7fd75c02e000 comp rx=0 tx=0).stop 2024-05-11T01:14:05.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 msgr2=0x7fd764071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:05.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 0x7fd764071a00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00e5e0 tx=0x7fd74c00eaa0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 shutdown_connections 2024-05-11T01:14:05.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fd7440762d0 0x7fd744078790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd76415f4d0 0x7fd764071a00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7640d32c0 0x7fd7640714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 --2- 172.21.15.52:0/3849008439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76406dfa0 0x7fd76414f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:05.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 >> 172.21.15.52:0/3849008439 conn(0x7fd76409ba20 msgr2=0x7fd7640681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:05.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 shutdown_connections 2024-05-11T01:14:05.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05.568+0000 7fd76ac09640 1 -- 172.21.15.52:0/3849008439 wait complete. 2024-05-11T01:14:05.578 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T01:14:05.578 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T01:14:05.578 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-05-11T01:14:05.578 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T01:14:05.578 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T01:14:05.578 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T01:14:05.578 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T01:14:05.579 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T01:14:05.579 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T01:14:05.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:05.579 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-05-11T01:14:05.595 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T01:14:05.596 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 41996sec preferred_lft 41996sec 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 1e:e6:90:cd:c4:f4 brd ff:ff:ff:ff:ff:ff 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:14:05.597 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::8c71:f4ff:fe68:49bf/64 scope link 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run.smithi052.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 1e:e6:90:cd:c4:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::1ce6:90ff:fecd:c4f4/64 scope link 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 72:00:c1:6e:a1:e0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::7000:c1ff:fe6e:a1e0/64 scope link 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-05-11T01:14:05.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:05.599 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T01:14:05.660 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-05-11T01:14:05.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:05.662 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-05-11T01:14:05.719 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-05-11T01:14:05.720 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T01:14:05.720 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-11T01:14:05.720 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-11T01:14:05.720 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T01:14:05.720 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T01:14:05.720 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-05-11T01:14:05.720 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-11T01:14:05.720 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-11T01:14:05.721 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-11T01:14:05.721 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-05-11T01:14:05.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T01:14:05.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:14:05.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:05.849 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-11T01:14:05.865 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T01:14:05.865 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T01:14:05.865 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T01:14:05.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:05.866 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-05-11T01:14:05.932 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-11T01:14:05.932 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:14:05.932 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T01:14:05.988 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T01:14:05.988 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T01:14:05.988 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T01:14:06.043 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-11T01:14:06.110 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:14:06.166 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-11T01:14:06.166 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-11T01:14:06.167 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:14:06.168 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:14:06.169 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:14:06.169 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:14:06.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:06.170 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-11T01:14:06.225 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-11T01:14:06.226 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-11T01:14:06.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-05-11T01:14:06.294 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T01:14:06.352 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-05-11T01:14:06.352 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-05-11T01:14:06.352 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-05-11T01:14:06.352 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-05-11T01:14:06.352 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-05-11T01:14:06.352 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-05-11T01:14:06.352 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-05-11T01:14:06.352 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-05-11T01:14:06.352 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T01:14:06.353 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:14:06.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:06.354 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-05-11T01:14:06.355 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:2024-05-11T01:14:06.354+0000 7f5b5dad2480 -1 init, newargv = 0x56411d07f0d0 newargc=15 2024-05-11T01:14:06.355 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[54784]: starting ceph client 2024-05-11T01:14:06.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[54784]: starting fuse 2024-05-11T01:14:06.371 INFO:teuthology.orchestra.run.smithi052.stdout:40 2024-05-11T01:14:06.371 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-11T01:14:06.371 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-05-11T01:14:06.371 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-05-11T01:14:06.371 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-05-11T01:14:06.371 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-05-11T01:14:06.371 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-05-11T01:14:06.371 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:14:06.371 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-05-11T01:14:06.371 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-05-11T01:14:06.372 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T01:14:06.373 DEBUG:teuthology.orchestra.run.smithi052:> 2024-05-11T01:14:06.373 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.0")) 2024-05-11T01:14:06.373 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-05-11T01:14:06.493 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.0.54784.asok 2024-05-11T01:14:06.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T01:14:06.503 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54784.asok 2024-05-11T01:14:06.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:06.504 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54784.asok status 2024-05-11T01:14:06.643 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:14:06.643 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-05-11T01:14:06.643 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "10380055a1fc5e7d3b37b252af7045f5c2c8ba29", 2024-05-11T01:14:06.643 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-3570-g10380055 (10380055a1fc5e7d3b37b252af7045f5c2c8ba29) squid (dev)", 2024-05-11T01:14:06.643 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "0", 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "54784", 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 5805, 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 5805 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-11T01:14:06.644 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1670320463 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1670320463 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.5805 192.168.0.1:0/1670320463", 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/1670320463", 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 24, 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 61, 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-05-11T01:14:06.645 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-05-11T01:14:06.646 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:14:06.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:06.652 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-05-11T01:14:06.652 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-11T01:14:06.668 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-05-11T01:14:06.668 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-11T01:14:06.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:14:06.669 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T01:14:06.735 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-11T01:14:06.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.882+0000 7efe6b789640 1 Processor -- start 2024-05-11T01:14:06.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.882+0000 7efe6b789640 1 -- start start 2024-05-11T01:14:06.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe6b789640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640703b0 0x7efe64160a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:06.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe6b789640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe64160fb0 0x7efe6415fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe6b789640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe640cf200 con 0x7efe64072a00 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe6b789640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7efe640cf4e0 con 0x7efe64160fb0 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe6b789640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efe640cf7c0 con 0x7efe640703b0 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe69525640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7efe64072a00 0x7efe64072de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:55264/0 (socket says 172.21.15.52:55264) 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe69525640 1 -- 172.21.15.52:0/971068039 learned_addr learned my addr 172.21.15.52:0/971068039 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe68d24640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640703b0 0x7efe64160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe69d26640 1 --2- 172.21.15.52:0/971068039 >> v2:172.21.15.119:3300/0 conn(0x7efe64160fb0 0x7efe6415fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2756716898 0 0) 0x7efe640cf200 con 0x7efe64072a00 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe50003520 con 0x7efe64072a00 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3856640251 0 0) 0x7efe50003520 con 0x7efe64072a00 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640703b0 msgr2=0x7efe64160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640703b0 0x7efe64160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:06.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 >> v2:172.21.15.119:3300/0 conn(0x7efe64160fb0 msgr2=0x7efe6415fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 --2- 172.21.15.52:0/971068039 >> v2:172.21.15.119:3300/0 conn(0x7efe64160fb0 0x7efe6415fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6415f9c0 con 0x7efe64072a00 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7efe60001a70 con 0x7efe64072a00 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 >> v1:172.21.15.52:6789/0 conn(0x7efe64072a00 legacy=0x7efe64072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640703b0 0x7efe50003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.883+0000 7efe5bfff640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe50003fd0 0x7efe50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe5bfff640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe50004d40 0x7efe500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe640cf4e0 con 0x7efe50004d40 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe640cf7c0 con 0x7efe50003fd0 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efe5000ba10 con 0x7efe640703b0 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe69d26640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe50004d40 0x7efe500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe69525640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640703b0 0x7efe50003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe68d24640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe50003fd0 0x7efe50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe69d26640 1 -- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640703b0 msgr2=0x7efe50003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe69d26640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640703b0 0x7efe50003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe69d26640 1 -- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe50003fd0 msgr2=0x7efe50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe69d26640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe50003fd0 0x7efe50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe69d26640 1 -- 172.21.15.52:0/971068039 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe50003520 con 0x7efe50004d40 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.884+0000 7efe68d24640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe50003fd0 0x7efe50004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe69d26640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe50004d40 0x7efe500051c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009a90 tx=0x7efe50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44411eff8db100f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c036070 con 0x7efe50004d40 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c031400 con 0x7efe50004d40 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe5bfff640 1 -- 172.21.15.52:0/971068039 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c009690 con 0x7efe50004d40 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe6b789640 1 -- 172.21.15.52:0/971068039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe50004d40 msgr2=0x7efe500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe6b789640 1 --2- 172.21.15.52:0/971068039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe50004d40 0x7efe500051c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009a90 tx=0x7efe50005830 comp rx=0 tx=0).stop 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe69d26640 1 -- 172.21.15.52:0/971068039 reap_dead start 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe6b789640 1 -- 172.21.15.52:0/971068039 shutdown_connections 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe6b789640 1 -- 172.21.15.52:0/971068039 >> 172.21.15.52:0/971068039 conn(0x7efe640760c0 msgr2=0x7efe6406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:14:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe6b789640 1 -- 172.21.15.52:0/971068039 shutdown_connections 2024-05-11T01:14:06.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.885+0000 7efe6b789640 1 -- 172.21.15.52:0/971068039 wait complete. 2024-05-11T01:14:06.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.886+0000 7efe6b789640 1 Processor -- start 2024-05-11T01:14:06.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe6b789640 1 -- start start 2024-05-11T01:14:06.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe6b789640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe50004d40 0x7efe640b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:06.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe6b789640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640b4e70 0x7efe6415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:06.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe6b789640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6415d6b0 0x7efe6415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:06.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe6b789640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe640d1340 con 0x7efe6415d6b0 2024-05-11T01:14:06.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe6b789640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe640cfaa0 con 0x7efe50004d40 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe6b789640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efe640d14c0 con 0x7efe640b4e70 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe69d26640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6415d6b0 0x7efe6415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe69d26640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6415d6b0 0x7efe6415db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49694/0 (socket says 172.21.15.52:49694) 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe69d26640 1 -- 172.21.15.52:0/1940078786 learned_addr learned my addr 172.21.15.52:0/1940078786 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe68d24640 1 --2- 172.21.15.52:0/1940078786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640b4e70 0x7efe6415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.887+0000 7efe69525640 1 --2- 172.21.15.52:0/1940078786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe50004d40 0x7efe640b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe69d26640 1 -- 172.21.15.52:0/1940078786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640b4e70 msgr2=0x7efe6415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe69d26640 1 --2- 172.21.15.52:0/1940078786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe640b4e70 0x7efe6415d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe69d26640 1 -- 172.21.15.52:0/1940078786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe50004d40 msgr2=0x7efe640b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe69d26640 1 --2- 172.21.15.52:0/1940078786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe50004d40 0x7efe640b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe69d26640 1 -- 172.21.15.52:0/1940078786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe64158f80 con 0x7efe6415d6b0 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe69d26640 1 --2- 172.21.15.52:0/1940078786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6415d6b0 0x7efe6415db30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009bb0 tx=0x7efe5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:06.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe59ffb640 1 -- 172.21.15.52:0/1940078786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c036040 con 0x7efe6415d6b0 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe59ffb640 1 -- 172.21.15.52:0/1940078786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c041070 con 0x7efe6415d6b0 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe59ffb640 1 -- 172.21.15.52:0/1940078786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c003a50 con 0x7efe6415d6b0 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe6b789640 1 -- 172.21.15.52:0/1940078786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe64147e90 con 0x7efe6415d6b0 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.888+0000 7efe6b789640 1 -- 172.21.15.52:0/1940078786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe641594e0 con 0x7efe6415d6b0 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.889+0000 7efe6b789640 1 -- 172.21.15.52:0/1940078786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe640cf200 con 0x7efe6415d6b0 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.890+0000 7efe59ffb640 1 -- 172.21.15.52:0/1940078786 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7efe5c003bf0 con 0x7efe6415d6b0 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.890+0000 7efe59ffb640 1 --2- 172.21.15.52:0/1940078786 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efe48076300 0x7efe480787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.890+0000 7efe59ffb640 1 -- 172.21.15.52:0/1940078786 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6265+0+0 (secure 0 0 0) 0x7efe5c0b6fb0 con 0x7efe6415d6b0 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.891+0000 7efe69525640 1 --2- 172.21.15.52:0/1940078786 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efe48076300 0x7efe480787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:14:06.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.891+0000 7efe69525640 1 --2- 172.21.15.52:0/1940078786 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efe48076300 0x7efe480787c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe60001390 tx=0x7efe60006040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:14:06.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:06.895+0000 7efe59ffb640 1 -- 172.21.15.52:0/1940078786 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efe640cf200 con 0x7efe6415d6b0 2024-05-11T01:14:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:14:07.116+0000 7efe6b789640 1 -- 172.21.15.52:0/1940078786 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7efe64069780 con 0x7efe48076300 2024-05-11T01:14:13.841 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:14:13.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:14:43.963 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:14:43.966 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:15:14.102 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:15:14.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:15:44.240 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:15:44.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:16:06.807 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-05-11T01:16:06.810 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ERROR 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner: 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner:ERROR: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/cephfs/test_snap_schedules.py", line 103, in setUp 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner: self._create_or_reuse_test_volume() 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/cephfs/test_snap_schedules.py", line 79, in _create_or_reuse_test_volume 2024-05-11T01:16:06.811 INFO:tasks.cephfs_test_runner: result = json.loads(self._fs_cmd("volume", "ls")) 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/cephfs/test_snap_schedules.py", line 63, in _fs_cmd 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: r.wait() 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi052 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls' 2024-05-11T01:16:06.812 INFO:tasks.cephfs_test_runner: 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner:Ran 2 tests in 880.568s 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner: 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner: 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner:ERROR: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/cephfs/test_snap_schedules.py", line 103, in setUp 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner: self._create_or_reuse_test_volume() 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/cephfs/test_snap_schedules.py", line 79, in _create_or_reuse_test_volume 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner: result = json.loads(self._fs_cmd("volume", "ls")) 2024-05-11T01:16:06.813 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/cephfs/test_snap_schedules.py", line 63, in _fs_cmd 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: r.wait() 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi052 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls' 2024-05-11T01:16:06.814 INFO:tasks.cephfs_test_runner: 2024-05-11T01:16:06.815 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-05-11T01:16:07.057 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=483982b109f5420ab2c1fbc42157ebf3 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_10380055a1fc5e7d3b37b252af7045f5c2c8ba29/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-05-11T01:16:07.059 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-11T01:16:07.071 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-11T01:16:07.086 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-11T01:16:07.086 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:16:07.086 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:16:07.109 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:16:07.109 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:16:07.169 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:16:07.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.324+0000 7fc6418e8640 1 Processor -- start 2024-05-11T01:16:07.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.324+0000 7fc6418e8640 1 -- start start 2024-05-11T01:16:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.325+0000 7fc6418e8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc63c0afb30 0x7fc63c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.325+0000 7fc6418e8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc63c0aea90 con 0x7fc63c072ce0 2024-05-11T01:16:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.325+0000 7fc6418e8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc63c0aed70 con 0x7fc63c073600 2024-05-11T01:16:07.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.325+0000 7fc6418e8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc63c06e760 con 0x7fc63c0afb30 2024-05-11T01:16:07.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.325+0000 7fc63affd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fc63c072ce0 0x7fc63c06e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:44886/0 (socket says 172.21.15.52:44886) 2024-05-11T01:16:07.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.325+0000 7fc63affd640 1 -- 172.21.15.52:0/1345917212 learned_addr learned my addr 172.21.15.52:0/1345917212 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:07.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1388873069 0 0) 0x7fc63c0aea90 con 0x7fc63c072ce0 2024-05-11T01:16:07.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc624003520 con 0x7fc63c072ce0 2024-05-11T01:16:07.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc63a7fc640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc63c0afb30 0x7fc63c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2089365878 0 0) 0x7fc624003520 con 0x7fc63c072ce0 2024-05-11T01:16:07.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc63c0afb30 msgr2=0x7fc63c0730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:07.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc63c0afb30 0x7fc63c0730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 >> v1:172.21.15.119:6789/0 conn(0x7fc63c073600 legacy=0x7fc63c165b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c0af050 con 0x7fc63c072ce0 2024-05-11T01:16:07.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7fc62c0016f0 con 0x7fc63c072ce0 2024-05-11T01:16:07.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 >> v1:172.21.15.52:6789/0 conn(0x7fc63c072ce0 legacy=0x7fc63c06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc624003a30 0x7fc624003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc624004800 0x7fc624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc63c0afb30 0x7fc624005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c06e760 con 0x7fc624004800 2024-05-11T01:16:07.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc62400baf0 con 0x7fc624003a30 2024-05-11T01:16:07.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc62400df00 con 0x7fc63c0afb30 2024-05-11T01:16:07.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc63a7fc640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc624004800 0x7fc624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc63affd640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc624003a30 0x7fc624003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc63b7fe640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc63c0afb30 0x7fc624005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc63a7fc640 1 -- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc63c0afb30 msgr2=0x7fc624005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:07.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc63a7fc640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc63c0afb30 0x7fc624005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc63a7fc640 1 -- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc624003a30 msgr2=0x7fc624003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:07.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc63a7fc640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc624003a30 0x7fc624003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.326+0000 7fc63a7fc640 1 -- 172.21.15.52:0/1345917212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc624003520 con 0x7fc624004800 2024-05-11T01:16:07.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.327+0000 7fc63a7fc640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc624004800 0x7fc624004c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc630009ac0 tx=0x7fc624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37111a71aba441d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:07.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.327+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630036070 con 0x7fc624004800 2024-05-11T01:16:07.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.327+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc630031400 con 0x7fc624004800 2024-05-11T01:16:07.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.327+0000 7fc639ffb640 1 -- 172.21.15.52:0/1345917212 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630009690 con 0x7fc624004800 2024-05-11T01:16:07.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.328+0000 7fc6418e8640 1 -- 172.21.15.52:0/1345917212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc624004800 msgr2=0x7fc624004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.328+0000 7fc6418e8640 1 --2- 172.21.15.52:0/1345917212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc624004800 0x7fc624004c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc630009ac0 tx=0x7fc624005830 comp rx=0 tx=0).stop 2024-05-11T01:16:07.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.328+0000 7fc63b7fe640 1 -- 172.21.15.52:0/1345917212 reap_dead start 2024-05-11T01:16:07.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.328+0000 7fc6418e8640 1 -- 172.21.15.52:0/1345917212 shutdown_connections 2024-05-11T01:16:07.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.328+0000 7fc6418e8640 1 -- 172.21.15.52:0/1345917212 >> 172.21.15.52:0/1345917212 conn(0x7fc63c0b2a00 msgr2=0x7fc63c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:07.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.328+0000 7fc6418e8640 1 -- 172.21.15.52:0/1345917212 shutdown_connections 2024-05-11T01:16:07.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.328+0000 7fc6418e8640 1 -- 172.21.15.52:0/1345917212 wait complete. 2024-05-11T01:16:07.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.329+0000 7fc6418e8640 1 Processor -- start 2024-05-11T01:16:07.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.329+0000 7fc6418e8640 1 -- start start 2024-05-11T01:16:07.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc6418e8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc624004800 0x7fc63c145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc6418e8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc63c1e4530 0x7fc63c148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc6418e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc63c15e3e0 0x7fc63c1452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc6418e8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c0af610 con 0x7fc63c1e4530 2024-05-11T01:16:07.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc6418e8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c073a50 con 0x7fc63c15e3e0 2024-05-11T01:16:07.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc6418e8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fc63c09cfc0 con 0x7fc624004800 2024-05-11T01:16:07.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc63a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc63c1e4530 0x7fc63c148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc63a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc63c1e4530 0x7fc63c148a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55260/0 (socket says 172.21.15.52:55260) 2024-05-11T01:16:07.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc63a7fc640 1 -- 172.21.15.52:0/2383471062 learned_addr learned my addr 172.21.15.52:0/2383471062 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:07.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.330+0000 7fc63b7fe640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc63c15e3e0 0x7fc63c1452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc63affd640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc624004800 0x7fc63c145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc63a7fc640 1 -- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc624004800 msgr2=0x7fc63c145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc63a7fc640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc624004800 0x7fc63c145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc63a7fc640 1 -- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc63c15e3e0 msgr2=0x7fc63c1452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc63a7fc640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc63c15e3e0 0x7fc63c1452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc63a7fc640 1 -- 172.21.15.52:0/2383471062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c148f60 con 0x7fc63c1e4530 2024-05-11T01:16:07.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc63affd640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc624004800 0x7fc63c145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:07.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc63a7fc640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc63c1e4530 0x7fc63c148a20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc630009ac0 tx=0x7fc630004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:07.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc623fff640 1 -- 172.21.15.52:0/2383471062 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630040020 con 0x7fc63c1e4530 2024-05-11T01:16:07.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc623fff640 1 -- 172.21.15.52:0/2383471062 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc630003bc0 con 0x7fc63c1e4530 2024-05-11T01:16:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63c1e49f0 con 0x7fc63c1e4530 2024-05-11T01:16:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc623fff640 1 -- 172.21.15.52:0/2383471062 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630036040 con 0x7fc63c1e4530 2024-05-11T01:16:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.331+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63c1e4fb0 con 0x7fc63c1e4530 2024-05-11T01:16:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.332+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63c069780 con 0x7fc63c1e4530 2024-05-11T01:16:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.333+0000 7fc623fff640 1 -- 172.21.15.52:0/2383471062 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fc630043020 con 0x7fc63c1e4530 2024-05-11T01:16:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.333+0000 7fc623fff640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6280762d0 0x7fc628078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.334+0000 7fc623fff640 1 -- 172.21.15.52:0/2383471062 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7fc6300b7c10 con 0x7fc63c1e4530 2024-05-11T01:16:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.334+0000 7fc63affd640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6280762d0 0x7fc628078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.334+0000 7fc63affd640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6280762d0 0x7fc628078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc62c007d90 tx=0x7fc62c009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:07.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.338+0000 7fc623fff640 1 -- 172.21.15.52:0/2383471062 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc63004a030 con 0x7fc63c1e4530 2024-05-11T01:16:07.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.543+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc63c0b1ca0 con 0x7fc63c1e4530 2024-05-11T01:16:07.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.543+0000 7fc623fff640 1 -- 172.21.15.52:0/2383471062 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7fc63c148f60 con 0x7fc63c1e4530 2024-05-11T01:16:07.542 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:16:07.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6280762d0 msgr2=0x7fc628078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6280762d0 0x7fc628078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc62c007d90 tx=0x7fc62c009040 comp rx=0 tx=0).stop 2024-05-11T01:16:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc63c1e4530 msgr2=0x7fc63c148a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc63c1e4530 0x7fc63c148a20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc630009ac0 tx=0x7fc630004730 comp rx=0 tx=0).stop 2024-05-11T01:16:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 shutdown_connections 2024-05-11T01:16:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fc6280762d0 0x7fc628078790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc63c15e3e0 0x7fc63c1452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc63c1e4530 0x7fc63c148a20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 --2- 172.21.15.52:0/2383471062 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc624004800 0x7fc63c145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 >> 172.21.15.52:0/2383471062 conn(0x7fc63c0b2a00 msgr2=0x7fc63c0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 shutdown_connections 2024-05-11T01:16:07.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.545+0000 7fc6418e8640 1 -- 172.21.15.52:0/2383471062 wait complete. 2024-05-11T01:16:07.555 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:16:07.555 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:16:07.571 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T01:16:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.717+0000 7f5ae64ea640 1 Processor -- start 2024-05-11T01:16:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.717+0000 7f5ae64ea640 1 -- start start 2024-05-11T01:16:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.718+0000 7f5ae64ea640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5ae01576a0 0x7f5ae00add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.718+0000 7f5ae64ea640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ae00b0060 0x7f5ae00733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.718+0000 7f5ae64ea640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ae0160c50 con 0x7f5ae00ae290 2024-05-11T01:16:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.718+0000 7f5ae64ea640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f5ae0160f30 con 0x7f5ae01576a0 2024-05-11T01:16:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.718+0000 7f5ae64ea640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae0161210 con 0x7f5ae00b0060 2024-05-11T01:16:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.718+0000 7f5adffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5ae01576a0 0x7f5ae00add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.719+0000 7f5adffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f5ae01576a0 0x7f5ae00add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55268/0 (socket says 172.21.15.52:55268) 2024-05-11T01:16:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.719+0000 7f5adffff640 1 -- 172.21.15.52:0/1907151507 learned_addr learned my addr 172.21.15.52:0/1907151507 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.719+0000 7f5adffff640 1 -- 172.21.15.52:0/1907151507 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ae00b0060 msgr2=0x7f5ae00733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.719+0000 7f5adffff640 1 --2- 172.21.15.52:0/1907151507 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ae00b0060 0x7f5ae00733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.719+0000 7f5adffff640 1 -- 172.21.15.52:0/1907151507 >> v1:172.21.15.119:6789/0 conn(0x7f5ae00ae290 legacy=0x7f5ae00af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.719+0000 7f5adffff640 1 -- 172.21.15.52:0/1907151507 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae01614f0 con 0x7f5ae01576a0 2024-05-11T01:16:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.719+0000 7f5adffff640 1 --2- 172.21.15.52:0/1907151507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae01576a0 0x7f5ae00add50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0069230 tx=0x7f5ad002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8fc4fd695dfde2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5adeffd640 1 -- 172.21.15.52:0/1907151507 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0037070 con 0x7f5ae01576a0 2024-05-11T01:16:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5adeffd640 1 -- 172.21.15.52:0/1907151507 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad0032440 con 0x7f5ae01576a0 2024-05-11T01:16:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5adeffd640 1 -- 172.21.15.52:0/1907151507 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0032810 con 0x7f5ae01576a0 2024-05-11T01:16:07.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5ae64ea640 1 -- 172.21.15.52:0/1907151507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae01576a0 msgr2=0x7f5ae00add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5ae64ea640 1 --2- 172.21.15.52:0/1907151507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae01576a0 0x7f5ae00add50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0069230 tx=0x7f5ad002a180 comp rx=0 tx=0).stop 2024-05-11T01:16:07.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5ae64ea640 1 -- 172.21.15.52:0/1907151507 shutdown_connections 2024-05-11T01:16:07.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5ae64ea640 1 --2- 172.21.15.52:0/1907151507 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ae00b0060 0x7f5ae00733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5ae64ea640 1 --2- 172.21.15.52:0/1907151507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae01576a0 0x7f5ae00add50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5ae64ea640 1 -- 172.21.15.52:0/1907151507 >> 172.21.15.52:0/1907151507 conn(0x7f5ae00b2a00 msgr2=0x7f5ae00b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:07.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5ae64ea640 1 -- 172.21.15.52:0/1907151507 shutdown_connections 2024-05-11T01:16:07.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.720+0000 7f5ae64ea640 1 -- 172.21.15.52:0/1907151507 wait complete. 2024-05-11T01:16:07.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.721+0000 7f5ae64ea640 1 Processor -- start 2024-05-11T01:16:07.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.722+0000 7f5ae64ea640 1 -- start start 2024-05-11T01:16:07.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.722+0000 7f5ae64ea640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ae00ae290 0x7f5ae014a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.722+0000 7f5ae64ea640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae00b0060 0x7f5ae0155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.722+0000 7f5ae64ea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae01576a0 0x7f5ae01f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.722+0000 7f5ae64ea640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae015e350 con 0x7f5ae00b0060 2024-05-11T01:16:07.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.722+0000 7f5ae64ea640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae015e1d0 con 0x7f5ae01576a0 2024-05-11T01:16:07.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.722+0000 7f5ae64ea640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae015e4d0 con 0x7f5ae00ae290 2024-05-11T01:16:07.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.722+0000 7f5adf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae00b0060 0x7f5ae0155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae00b0060 0x7f5ae0155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55270/0 (socket says 172.21.15.52:55270) 2024-05-11T01:16:07.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adf7fe640 1 -- 172.21.15.52:0/2107700762 learned_addr learned my addr 172.21.15.52:0/2107700762 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:07.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adffff640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ae00ae290 0x7f5ae014a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5ae4a87640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae01576a0 0x7f5ae01f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adf7fe640 1 -- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ae00ae290 msgr2=0x7f5ae014a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adf7fe640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ae00ae290 0x7f5ae014a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adf7fe640 1 -- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae01576a0 msgr2=0x7f5ae01f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adf7fe640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae01576a0 0x7f5ae01f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adf7fe640 1 -- 172.21.15.52:0/2107700762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae00b4390 con 0x7f5ae00b0060 2024-05-11T01:16:07.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adf7fe640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae00b0060 0x7f5ae0155d00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c430 tx=0x7f5ad400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adcff9640 1 -- 172.21.15.52:0/2107700762 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4011790 con 0x7f5ae00b0060 2024-05-11T01:16:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adcff9640 1 -- 172.21.15.52:0/2107700762 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad4017070 con 0x7f5ae00b0060 2024-05-11T01:16:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae00b4670 con 0x7f5ae00b0060 2024-05-11T01:16:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5adcff9640 1 -- 172.21.15.52:0/2107700762 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4011b70 con 0x7f5ae00b0060 2024-05-11T01:16:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.723+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae00b4c30 con 0x7f5ae00b0060 2024-05-11T01:16:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.724+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae0069780 con 0x7f5ae00b0060 2024-05-11T01:16:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.725+0000 7f5adcff9640 1 -- 172.21.15.52:0/2107700762 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f5ad4011d10 con 0x7f5ae00b0060 2024-05-11T01:16:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.725+0000 7f5adcff9640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5acc0762d0 0x7f5acc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.726+0000 7f5adcff9640 1 -- 172.21.15.52:0/2107700762 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f5ad4097630 con 0x7f5ae00b0060 2024-05-11T01:16:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.726+0000 7f5adffff640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5acc0762d0 0x7f5acc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.726+0000 7f5adffff640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5acc0762d0 0x7f5acc078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0034000 tx=0x7f5ad002e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:07.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.730+0000 7f5adcff9640 1 -- 172.21.15.52:0/2107700762 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5ad4060640 con 0x7f5ae00b0060 2024-05-11T01:16:07.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.932+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ae009ec80 con 0x7f5ae00b0060 2024-05-11T01:16:07.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.932+0000 7f5adcff9640 1 -- 172.21.15.52:0/2107700762 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7f5ad4064940 con 0x7f5ae00b0060 2024-05-11T01:16:07.931 INFO:teuthology.orchestra.run.smithi052.stderr:listed 0 entries 2024-05-11T01:16:07.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5acc0762d0 msgr2=0x7f5acc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5acc0762d0 0x7f5acc078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0034000 tx=0x7f5ad002e000 comp rx=0 tx=0).stop 2024-05-11T01:16:07.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae00b0060 msgr2=0x7f5ae0155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:07.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae00b0060 0x7f5ae0155d00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c430 tx=0x7f5ad400c8f0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 shutdown_connections 2024-05-11T01:16:07.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f5acc0762d0 0x7f5acc078790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae01576a0 0x7f5ae01f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ae00b0060 0x7f5ae0155d00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 --2- 172.21.15.52:0/2107700762 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5ae00ae290 0x7f5ae014a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:07.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 >> 172.21.15.52:0/2107700762 conn(0x7f5ae00b2a00 msgr2=0x7f5ae015cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:07.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 shutdown_connections 2024-05-11T01:16:07.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:07.934+0000 7f5ae64ea640 1 -- 172.21.15.52:0/2107700762 wait complete. 2024-05-11T01:16:07.944 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-05-11T01:16:07.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:07.944 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-11T01:16:07.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[54784]: fuse finished with error 0 and tester_r 0 2024-05-11T01:16:07.971 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:14.072 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:16:14.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:14.073 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T01:16:14.093 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-11T01:16:14.094 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:16:14.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3484 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T01:16:14.154 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:16:14.154 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:16:14.175 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13678 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T01:16:14.175 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-11T01:16:14.175 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T01:16:14.234 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13678 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T01:16:14.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:14.234 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-05-11T01:16:14.294 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T01:16:14.294 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T01:16:14.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:14.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T01:16:14.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:14.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T01:16:14.395 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:16:14.398 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:16:14.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:14.435 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-05-11T01:16:14.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:14.554 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-05-11T01:16:14.575 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T01:16:14.575 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T01:16:14.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:14.576 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T01:16:14.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:14.646 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T01:16:14.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T01:16:14.716 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-05-11T01:16:14.824 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-11T01:16:14.833 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T01:16:14.834 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:16:14.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.942+0000 7f8a43064640 1 Processor -- start 2024-05-11T01:16:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.942+0000 7f8a43064640 1 -- start start 2024-05-11T01:16:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.943+0000 7f8a43064640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a3c0d3330 0x7f8a3c0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.943+0000 7f8a43064640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a3c0d1f90 con 0x7f8a3c0d2ea0 2024-05-11T01:16:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.943+0000 7f8a43064640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a3c0d2270 con 0x7f8a3c15f2d0 2024-05-11T01:16:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.943+0000 7f8a43064640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3c0d2550 con 0x7f8a3c0d3330 2024-05-11T01:16:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3bfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8a3c0d2ea0 0x7f8a3c0b2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33692/0 (socket says 172.21.15.52:33692) 2024-05-11T01:16:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3bfff640 1 -- 172.21.15.52:0/3878799477 learned_addr learned my addr 172.21.15.52:0/3878799477 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a41601640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a3c0d3330 0x7f8a3c0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3202468373 0 0) 0x7f8a3c0d1f90 con 0x7f8a3c0d2ea0 2024-05-11T01:16:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a28003520 con 0x7f8a3c0d2ea0 2024-05-11T01:16:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1461076233 0 0) 0x7f8a28003520 con 0x7f8a3c0d2ea0 2024-05-11T01:16:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a3c0d3330 msgr2=0x7f8a3c0b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a3c0d3330 0x7f8a3c0b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 >> v1:172.21.15.119:6789/0 conn(0x7f8a3c15f2d0 legacy=0x7f8a3c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c0d2830 con 0x7f8a3c0d2ea0 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f8a2c001a70 con 0x7f8a3c0d2ea0 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 >> v1:172.21.15.52:6789/0 conn(0x7f8a3c0d2ea0 legacy=0x7f8a3c0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a280039e0 0x7f8a28003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a3c0d3330 0x7f8a280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c0d2550 con 0x7f8a28004800 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2800ba70 con 0x7f8a280039e0 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2800bc70 con 0x7f8a3c0d3330 2024-05-11T01:16:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a3bfff640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.944+0000 7f8a40e00640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a280039e0 0x7f8a28003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.945+0000 7f8a41601640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a3c0d3330 0x7f8a280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.945+0000 7f8a3bfff640 1 -- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a3c0d3330 msgr2=0x7f8a280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.945+0000 7f8a3bfff640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a3c0d3330 0x7f8a280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.945+0000 7f8a3bfff640 1 -- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a280039e0 msgr2=0x7f8a28003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.945+0000 7f8a3bfff640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a280039e0 0x7f8a28003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.945+0000 7f8a3bfff640 1 -- 172.21.15.52:0/3878799477 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a28003520 con 0x7f8a28004800 2024-05-11T01:16:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.945+0000 7f8a3bfff640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00a880 tx=0x7f8a28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7a9d36e5e8077e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00a4d0 con 0x7f8a28004800 2024-05-11T01:16:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2c02f070 con 0x7f8a28004800 2024-05-11T01:16:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a3b7fe640 1 -- 172.21.15.52:0/3878799477 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c02bbc0 con 0x7f8a28004800 2024-05-11T01:16:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a43064640 1 -- 172.21.15.52:0/3878799477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a28004800 msgr2=0x7f8a28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a43064640 1 --2- 172.21.15.52:0/3878799477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00a880 tx=0x7f8a28005830 comp rx=0 tx=0).stop 2024-05-11T01:16:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a41601640 1 -- 172.21.15.52:0/3878799477 reap_dead start 2024-05-11T01:16:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a43064640 1 -- 172.21.15.52:0/3878799477 shutdown_connections 2024-05-11T01:16:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a43064640 1 -- 172.21.15.52:0/3878799477 >> 172.21.15.52:0/3878799477 conn(0x7f8a3c09ba20 msgr2=0x7f8a3c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a43064640 1 -- 172.21.15.52:0/3878799477 shutdown_connections 2024-05-11T01:16:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.946+0000 7f8a43064640 1 -- 172.21.15.52:0/3878799477 wait complete. 2024-05-11T01:16:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.948+0000 7f8a43064640 1 Processor -- start 2024-05-11T01:16:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.948+0000 7f8a43064640 1 -- start start 2024-05-11T01:16:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.948+0000 7f8a43064640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a28004800 0x7f8a3c153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.948+0000 7f8a43064640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a3c1593b0 0x7f8a3c1f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.948+0000 7f8a43064640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a3c157a10 0x7f8a3c160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.948+0000 7f8a43064640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c15f6b0 con 0x7f8a3c157a10 2024-05-11T01:16:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a43064640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c0d3740 con 0x7f8a3c1593b0 2024-05-11T01:16:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a43064640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3c15f830 con 0x7f8a28004800 2024-05-11T01:16:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a41601640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a3c157a10 0x7f8a3c160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a41601640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a3c157a10 0x7f8a3c160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43346/0 (socket says 172.21.15.52:43346) 2024-05-11T01:16:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a41601640 1 -- 172.21.15.52:0/604936290 learned_addr learned my addr 172.21.15.52:0/604936290 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a40e00640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a28004800 0x7f8a3c153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a3bfff640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a3c1593b0 0x7f8a3c1f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a41601640 1 -- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a28004800 msgr2=0x7f8a3c153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a41601640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a28004800 0x7f8a3c153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a41601640 1 -- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a3c1593b0 msgr2=0x7f8a3c1f91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a41601640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a3c1593b0 0x7f8a3c1f91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a41601640 1 -- 172.21.15.52:0/604936290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c1f8a00 con 0x7f8a3c157a10 2024-05-11T01:16:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a40e00640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a28004800 0x7f8a3c153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a41601640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a3c157a10 0x7f8a3c160c90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0d0a50 tx=0x7f8a34011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a397fa640 1 -- 172.21.15.52:0/604936290 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34004240 con 0x7f8a3c157a10 2024-05-11T01:16:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a397fa640 1 -- 172.21.15.52:0/604936290 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3401b070 con 0x7f8a3c157a10 2024-05-11T01:16:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c159870 con 0x7f8a3c157a10 2024-05-11T01:16:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a397fa640 1 -- 172.21.15.52:0/604936290 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400f770 con 0x7f8a3c157a10 2024-05-11T01:16:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.949+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1542a0 con 0x7f8a3c157a10 2024-05-11T01:16:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.950+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c069780 con 0x7f8a3c157a10 2024-05-11T01:16:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.951+0000 7f8a397fa640 1 -- 172.21.15.52:0/604936290 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f8a34012040 con 0x7f8a3c157a10 2024-05-11T01:16:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.952+0000 7f8a397fa640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a1c0762d0 0x7f8a1c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.952+0000 7f8a397fa640 1 -- 172.21.15.52:0/604936290 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f8a340976f0 con 0x7f8a3c157a10 2024-05-11T01:16:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.952+0000 7f8a40e00640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a1c0762d0 0x7f8a1c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.953+0000 7f8a40e00640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a1c0762d0 0x7f8a1c078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8a240011a0 tx=0x7f8a24009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:14.956+0000 7f8a397fa640 1 -- 172.21.15.52:0/604936290 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8a34060de0 con 0x7f8a3c157a10 2024-05-11T01:16:15.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.145+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a3c15e930 con 0x7f8a1c0762d0 2024-05-11T01:16:15.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.153+0000 7f8a397fa640 1 -- 172.21.15.52:0/604936290 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251840 (secure 0 0 0) 0x7f8a3c15e930 con 0x7f8a1c0762d0 2024-05-11T01:16:15.156 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:16:15.157 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:16:15.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a1c0762d0 msgr2=0x7f8a1c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a1c0762d0 0x7f8a1c078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8a240011a0 tx=0x7f8a24009040 comp rx=0 tx=0).stop 2024-05-11T01:16:15.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a3c157a10 msgr2=0x7f8a3c160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a3c157a10 0x7f8a3c160c90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0d0a50 tx=0x7f8a34011cf0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 shutdown_connections 2024-05-11T01:16:15.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a1c0762d0 0x7f8a1c078790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a3c157a10 0x7f8a3c160c90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a3c1593b0 0x7f8a3c1f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 --2- 172.21.15.52:0/604936290 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a28004800 0x7f8a3c153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 >> 172.21.15.52:0/604936290 conn(0x7f8a3c09ba20 msgr2=0x7f8a3c0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:15.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 shutdown_connections 2024-05-11T01:16:15.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.156+0000 7f8a43064640 1 -- 172.21.15.52:0/604936290 wait complete. 2024-05-11T01:16:15.167 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-05-11T01:16:13.739942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594175,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":174,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317588,"kb_used_data":7080,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754657132,"statfs":{"total":773094113280,"available":772768903168,"internally_reserved":0,"allocated":7249920,"data_stored":4059958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":317836849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003405"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854522+0000","last_change":"2024-05-11T01:13:59.434893+0000","last_active":"2024-05-11T01:14:46.854522+0000","last_peered":"2024-05-11T01:14:46.854522+0000","last_clean":"2024-05-11T01:14:46.854522+0000","last_became_active":"2024-05-11T01:13:59.434594+0000","last_became_peered":"2024-05-11T01:13:59.434594+0000","last_unstale":"2024-05-11T01:14:46.854522+0000","last_undegraded":"2024-05-11T01:14:46.854522+0000","last_fullsized":"2024-05-11T01:14:46.854522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:49:33.685964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854143+0000","last_change":"2024-05-11T01:13:59.446282+0000","last_active":"2024-05-11T01:14:46.854143+0000","last_peered":"2024-05-11T01:14:46.854143+0000","last_clean":"2024-05-11T01:14:46.854143+0000","last_became_active":"2024-05-11T01:13:59.445971+0000","last_became_peered":"2024-05-11T01:13:59.445971+0000","last_unstale":"2024-05-11T01:14:46.854143+0000","last_undegraded":"2024-05-11T01:14:46.854143+0000","last_fullsized":"2024-05-11T01:14:46.854143+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:50.869559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839269+0000","last_change":"2024-05-11T01:13:59.430661+0000","last_active":"2024-05-11T01:14:46.839269+0000","last_peered":"2024-05-11T01:14:46.839269+0000","last_clean":"2024-05-11T01:14:46.839269+0000","last_became_active":"2024-05-11T01:13:59.430408+0000","last_became_peered":"2024-05-11T01:13:59.430408+0000","last_unstale":"2024-05-11T01:14:46.839269+0000","last_undegraded":"2024-05-11T01:14:46.839269+0000","last_fullsized":"2024-05-11T01:14:46.839269+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:14:03.379607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854537+0000","last_change":"2024-05-11T01:13:59.448424+0000","last_active":"2024-05-11T01:14:46.854537+0000","last_peered":"2024-05-11T01:14:46.854537+0000","last_clean":"2024-05-11T01:14:46.854537+0000","last_became_active":"2024-05-11T01:13:59.446986+0000","last_became_peered":"2024-05-11T01:13:59.446986+0000","last_unstale":"2024-05-11T01:14:46.854537+0000","last_undegraded":"2024-05-11T01:14:46.854537+0000","last_fullsized":"2024-05-11T01:14:46.854537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853610+0000","last_change":"2024-05-11T01:13:59.446369+0000","last_active":"2024-05-11T01:14:46.853610+0000","last_peered":"2024-05-11T01:14:46.853610+0000","last_clean":"2024-05-11T01:14:46.853610+0000","last_became_active":"2024-05-11T01:13:59.445922+0000","last_became_peered":"2024-05-11T01:13:59.445922+0000","last_unstale":"2024-05-11T01:14:46.853610+0000","last_undegraded":"2024-05-11T01:14:46.853610+0000","last_fullsized":"2024-05-11T01:14:46.853610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:15.278361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837202+0000","last_change":"2024-05-11T01:13:59.435535+0000","last_active":"2024-05-11T01:14:46.837202+0000","last_peered":"2024-05-11T01:14:46.837202+0000","last_clean":"2024-05-11T01:14:46.837202+0000","last_became_active":"2024-05-11T01:13:59.434634+0000","last_became_peered":"2024-05-11T01:13:59.434634+0000","last_unstale":"2024-05-11T01:14:46.837202+0000","last_undegraded":"2024-05-11T01:14:46.837202+0000","last_fullsized":"2024-05-11T01:14:46.837202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:34:16.451030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854001+0000","last_change":"2024-05-11T01:13:59.434552+0000","last_active":"2024-05-11T01:14:46.854001+0000","last_peered":"2024-05-11T01:14:46.854001+0000","last_clean":"2024-05-11T01:14:46.854001+0000","last_became_active":"2024-05-11T01:13:59.433976+0000","last_became_peered":"2024-05-11T01:13:59.433976+0000","last_unstale":"2024-05-11T01:14:46.854001+0000","last_undegraded":"2024-05-11T01:14:46.854001+0000","last_fullsized":"2024-05-11T01:14:46.854001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853803+0000","last_change":"2024-05-11T01:13:59.447658+0000","last_active":"2024-05-11T01:14:46.853803+0000","last_peered":"2024-05-11T01:14:46.853803+0000","last_clean":"2024-05-11T01:14:46.853803+0000","last_became_active":"2024-05-11T01:13:59.446507+0000","last_became_peered":"2024-05-11T01:13:59.446507+0000","last_unstale":"2024-05-11T01:14:46.853803+0000","last_undegraded":"2024-05-11T01:14:46.853803+0000","last_fullsized":"2024-05-11T01:14:46.853803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:34:51.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854683+0000","last_change":"2024-05-11T01:13:59.449973+0000","last_active":"2024-05-11T01:14:46.854683+0000","last_peered":"2024-05-11T01:14:46.854683+0000","last_clean":"2024-05-11T01:14:46.854683+0000","last_became_active":"2024-05-11T01:13:59.449670+0000","last_became_peered":"2024-05-11T01:13:59.449670+0000","last_unstale":"2024-05-11T01:14:46.854683+0000","last_undegraded":"2024-05-11T01:14:46.854683+0000","last_fullsized":"2024-05-11T01:14:46.854683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854986+0000","last_change":"2024-05-11T01:13:59.450134+0000","last_active":"2024-05-11T01:14:46.854986+0000","last_peered":"2024-05-11T01:14:46.854986+0000","last_clean":"2024-05-11T01:14:46.854986+0000","last_became_active":"2024-05-11T01:13:59.449822+0000","last_became_peered":"2024-05-11T01:13:59.449822+0000","last_unstale":"2024-05-11T01:14:46.854986+0000","last_undegraded":"2024-05-11T01:14:46.854986+0000","last_fullsized":"2024-05-11T01:14:46.854986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.840088+0000","last_change":"2024-05-11T01:14:46.840088+0000","last_active":"2024-05-11T01:14:46.840088+0000","last_peered":"2024-05-11T01:14:46.840088+0000","last_clean":"2024-05-11T01:14:46.840088+0000","last_became_active":"2024-05-11T01:14:46.839837+0000","last_became_peered":"2024-05-11T01:14:46.839837+0000","last_unstale":"2024-05-11T01:14:46.840088+0000","last_undegraded":"2024-05-11T01:14:46.840088+0000","last_fullsized":"2024-05-11T01:14:46.840088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:38:11.978713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853737+0000","last_change":"2024-05-11T01:13:59.428513+0000","last_active":"2024-05-11T01:14:46.853737+0000","last_peered":"2024-05-11T01:14:46.853737+0000","last_clean":"2024-05-11T01:14:46.853737+0000","last_became_active":"2024-05-11T01:13:59.428202+0000","last_became_peered":"2024-05-11T01:13:59.428202+0000","last_unstale":"2024-05-11T01:14:46.853737+0000","last_undegraded":"2024-05-11T01:14:46.853737+0000","last_fullsized":"2024-05-11T01:14:46.853737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:23.500916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854223+0000","last_change":"2024-05-11T01:13:59.435402+0000","last_active":"2024-05-11T01:14:46.854223+0000","last_peered":"2024-05-11T01:14:46.854223+0000","last_clean":"2024-05-11T01:14:46.854223+0000","last_became_active":"2024-05-11T01:13:59.433324+0000","last_became_peered":"2024-05-11T01:13:59.433324+0000","last_unstale":"2024-05-11T01:14:46.854223+0000","last_undegraded":"2024-05-11T01:14:46.854223+0000","last_fullsized":"2024-05-11T01:14:46.854223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:36:46.612605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854315+0000","last_change":"2024-05-11T01:13:59.447113+0000","last_active":"2024-05-11T01:14:46.854315+0000","last_peered":"2024-05-11T01:14:46.854315+0000","last_clean":"2024-05-11T01:14:46.854315+0000","last_became_active":"2024-05-11T01:13:59.446349+0000","last_became_peered":"2024-05-11T01:13:59.446349+0000","last_unstale":"2024-05-11T01:14:46.854315+0000","last_undegraded":"2024-05-11T01:14:46.854315+0000","last_fullsized":"2024-05-11T01:14:46.854315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836869+0000","last_change":"2024-05-11T01:13:59.437811+0000","last_active":"2024-05-11T01:14:46.836869+0000","last_peered":"2024-05-11T01:14:46.836869+0000","last_clean":"2024-05-11T01:14:46.836869+0000","last_became_active":"2024-05-11T01:13:59.427975+0000","last_became_peered":"2024-05-11T01:13:59.427975+0000","last_unstale":"2024-05-11T01:14:46.836869+0000","last_undegraded":"2024-05-11T01:14:46.836869+0000","last_fullsized":"2024-05-11T01:14:46.836869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:05:27.108182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836920+0000","last_change":"2024-05-11T01:13:59.446666+0000","last_active":"2024-05-11T01:14:46.836920+0000","last_peered":"2024-05-11T01:14:46.836920+0000","last_clean":"2024-05-11T01:14:46.836920+0000","last_became_active":"2024-05-11T01:13:59.446396+0000","last_became_peered":"2024-05-11T01:13:59.446396+0000","last_unstale":"2024-05-11T01:14:46.836920+0000","last_undegraded":"2024-05-11T01:14:46.836920+0000","last_fullsized":"2024-05-11T01:14:46.836920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:55:23.003515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839156+0000","last_change":"2024-05-11T01:13:59.437261+0000","last_active":"2024-05-11T01:14:46.839156+0000","last_peered":"2024-05-11T01:14:46.839156+0000","last_clean":"2024-05-11T01:14:46.839156+0000","last_became_active":"2024-05-11T01:13:59.436992+0000","last_became_peered":"2024-05-11T01:13:59.436992+0000","last_unstale":"2024-05-11T01:14:46.839156+0000","last_undegraded":"2024-05-11T01:14:46.839156+0000","last_fullsized":"2024-05-11T01:14:46.839156+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:53.116569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839026+0000","last_change":"2024-05-11T01:13:59.445508+0000","last_active":"2024-05-11T01:14:46.839026+0000","last_peered":"2024-05-11T01:14:46.839026+0000","last_clean":"2024-05-11T01:14:46.839026+0000","last_became_active":"2024-05-11T01:13:59.445318+0000","last_became_peered":"2024-05-11T01:13:59.445318+0000","last_unstale":"2024-05-11T01:14:46.839026+0000","last_undegraded":"2024-05-11T01:14:46.839026+0000","last_fullsized":"2024-05-11T01:14:46.839026+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:21.154101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855679+0000","last_change":"2024-05-11T01:14:46.855679+0000","last_active":"2024-05-11T01:14:46.855679+0000","last_peered":"2024-05-11T01:14:46.855679+0000","last_clean":"2024-05-11T01:14:46.855679+0000","last_became_active":"2024-05-11T01:14:46.855320+0000","last_became_peered":"2024-05-11T01:14:46.855320+0000","last_unstale":"2024-05-11T01:14:46.855679+0000","last_undegraded":"2024-05-11T01:14:46.855679+0000","last_fullsized":"2024-05-11T01:14:46.855679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:48:36.304487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854538+0000","last_change":"2024-05-11T01:13:59.444231+0000","last_active":"2024-05-11T01:14:46.854538+0000","last_peered":"2024-05-11T01:14:46.854538+0000","last_clean":"2024-05-11T01:14:46.854538+0000","last_became_active":"2024-05-11T01:13:59.443762+0000","last_became_peered":"2024-05-11T01:13:59.443762+0000","last_unstale":"2024-05-11T01:14:46.854538+0000","last_undegraded":"2024-05-11T01:14:46.854538+0000","last_fullsized":"2024-05-11T01:14:46.854538+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:25:17.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837269+0000","last_change":"2024-05-11T01:13:59.437781+0000","last_active":"2024-05-11T01:14:46.837269+0000","last_peered":"2024-05-11T01:14:46.837269+0000","last_clean":"2024-05-11T01:14:46.837269+0000","last_became_active":"2024-05-11T01:13:59.437444+0000","last_became_peered":"2024-05-11T01:13:59.437444+0000","last_unstale":"2024-05-11T01:14:46.837269+0000","last_undegraded":"2024-05-11T01:14:46.837269+0000","last_fullsized":"2024-05-11T01:14:46.837269+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:31:12.326102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854442+0000","last_change":"2024-05-11T01:13:59.447343+0000","last_active":"2024-05-11T01:14:46.854442+0000","last_peered":"2024-05-11T01:14:46.854442+0000","last_clean":"2024-05-11T01:14:46.854442+0000","last_became_active":"2024-05-11T01:13:59.446779+0000","last_became_peered":"2024-05-11T01:13:59.446779+0000","last_unstale":"2024-05-11T01:14:46.854442+0000","last_undegraded":"2024-05-11T01:14:46.854442+0000","last_fullsized":"2024-05-11T01:14:46.854442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836924+0000","last_change":"2024-05-11T01:13:59.431582+0000","last_active":"2024-05-11T01:14:46.836924+0000","last_peered":"2024-05-11T01:14:46.836924+0000","last_clean":"2024-05-11T01:14:46.836924+0000","last_became_active":"2024-05-11T01:13:59.431309+0000","last_became_peered":"2024-05-11T01:13:59.431309+0000","last_unstale":"2024-05-11T01:14:46.836924+0000","last_undegraded":"2024-05-11T01:14:46.836924+0000","last_fullsized":"2024-05-11T01:14:46.836924+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837032+0000","last_change":"2024-05-11T01:13:59.433902+0000","last_active":"2024-05-11T01:14:46.837032+0000","last_peered":"2024-05-11T01:14:46.837032+0000","last_clean":"2024-05-11T01:14:46.837032+0000","last_became_active":"2024-05-11T01:13:59.433606+0000","last_became_peered":"2024-05-11T01:13:59.433606+0000","last_unstale":"2024-05-11T01:14:46.837032+0000","last_undegraded":"2024-05-11T01:14:46.837032+0000","last_fullsized":"2024-05-11T01:14:46.837032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837537+0000","last_change":"2024-05-11T01:13:59.436026+0000","last_active":"2024-05-11T01:14:46.837537+0000","last_peered":"2024-05-11T01:14:46.837537+0000","last_clean":"2024-05-11T01:14:46.837537+0000","last_became_active":"2024-05-11T01:13:59.435708+0000","last_became_peered":"2024-05-11T01:13:59.435708+0000","last_unstale":"2024-05-11T01:14:46.837537+0000","last_undegraded":"2024-05-11T01:14:46.837537+0000","last_fullsized":"2024-05-11T01:14:46.837537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854605+0000","last_change":"2024-05-11T01:13:59.441132+0000","last_active":"2024-05-11T01:14:46.854605+0000","last_peered":"2024-05-11T01:14:46.854605+0000","last_clean":"2024-05-11T01:14:46.854605+0000","last_became_active":"2024-05-11T01:13:59.440852+0000","last_became_peered":"2024-05-11T01:13:59.440852+0000","last_unstale":"2024-05-11T01:14:46.854605+0000","last_undegraded":"2024-05-11T01:14:46.854605+0000","last_fullsized":"2024-05-11T01:14:46.854605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:18:58.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839326+0000","last_change":"2024-05-11T01:13:59.436547+0000","last_active":"2024-05-11T01:14:46.839326+0000","last_peered":"2024-05-11T01:14:46.839326+0000","last_clean":"2024-05-11T01:14:46.839326+0000","last_became_active":"2024-05-11T01:13:59.436260+0000","last_became_peered":"2024-05-11T01:13:59.436260+0000","last_unstale":"2024-05-11T01:14:46.839326+0000","last_undegraded":"2024-05-11T01:14:46.839326+0000","last_fullsized":"2024-05-11T01:14:46.839326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:27.176219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853867+0000","last_change":"2024-05-11T01:13:59.425177+0000","last_active":"2024-05-11T01:14:46.853867+0000","last_peered":"2024-05-11T01:14:46.853867+0000","last_clean":"2024-05-11T01:14:46.853867+0000","last_became_active":"2024-05-11T01:13:59.424328+0000","last_became_peered":"2024-05-11T01:13:59.424328+0000","last_unstale":"2024-05-11T01:14:46.853867+0000","last_undegraded":"2024-05-11T01:14:46.853867+0000","last_fullsized":"2024-05-11T01:14:46.853867+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:29.739616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837190+0000","last_change":"2024-05-11T01:13:59.436551+0000","last_active":"2024-05-11T01:14:46.837190+0000","last_peered":"2024-05-11T01:14:46.837190+0000","last_clean":"2024-05-11T01:14:46.837190+0000","last_became_active":"2024-05-11T01:13:59.436257+0000","last_became_peered":"2024-05-11T01:13:59.436257+0000","last_unstale":"2024-05-11T01:14:46.837190+0000","last_undegraded":"2024-05-11T01:14:46.837190+0000","last_fullsized":"2024-05-11T01:14:46.837190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:37:44.925797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838948+0000","last_change":"2024-05-11T01:13:59.435398+0000","last_active":"2024-05-11T01:14:46.838948+0000","last_peered":"2024-05-11T01:14:46.838948+0000","last_clean":"2024-05-11T01:14:46.838948+0000","last_became_active":"2024-05-11T01:13:59.435148+0000","last_became_peered":"2024-05-11T01:13:59.435148+0000","last_unstale":"2024-05-11T01:14:46.838948+0000","last_undegraded":"2024-05-11T01:14:46.838948+0000","last_fullsized":"2024-05-11T01:14:46.838948+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:14:43.435242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837071+0000","last_change":"2024-05-11T01:13:59.438524+0000","last_active":"2024-05-11T01:14:46.837071+0000","last_peered":"2024-05-11T01:14:46.837071+0000","last_clean":"2024-05-11T01:14:46.837071+0000","last_became_active":"2024-05-11T01:13:59.438262+0000","last_became_peered":"2024-05-11T01:13:59.438262+0000","last_unstale":"2024-05-11T01:14:46.837071+0000","last_undegraded":"2024-05-11T01:14:46.837071+0000","last_fullsized":"2024-05-11T01:14:46.837071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:44:46.930438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854377+0000","last_change":"2024-05-11T01:13:59.432090+0000","last_active":"2024-05-11T01:14:46.854377+0000","last_peered":"2024-05-11T01:14:46.854377+0000","last_clean":"2024-05-11T01:14:46.854377+0000","last_became_active":"2024-05-11T01:13:59.431691+0000","last_became_peered":"2024-05-11T01:13:59.431691+0000","last_unstale":"2024-05-11T01:14:46.854377+0000","last_undegraded":"2024-05-11T01:14:46.854377+0000","last_fullsized":"2024-05-11T01:14:46.854377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:11:10.857350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836734+0000","last_change":"2024-05-11T01:13:59.440442+0000","last_active":"2024-05-11T01:14:46.836734+0000","last_peered":"2024-05-11T01:14:46.836734+0000","last_clean":"2024-05-11T01:14:46.836734+0000","last_became_active":"2024-05-11T01:13:59.440107+0000","last_became_peered":"2024-05-11T01:13:59.440107+0000","last_unstale":"2024-05-11T01:14:46.836734+0000","last_undegraded":"2024-05-11T01:14:46.836734+0000","last_fullsized":"2024-05-11T01:14:46.836734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:58:40.948977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837590+0000","last_change":"2024-05-11T01:13:58.440158+0000","last_active":"2024-05-11T01:14:46.837590+0000","last_peered":"2024-05-11T01:14:46.837590+0000","last_clean":"2024-05-11T01:14:46.837590+0000","last_became_active":"2024-05-11T01:13:58.439841+0000","last_became_peered":"2024-05-11T01:13:58.439841+0000","last_unstale":"2024-05-11T01:14:46.837590+0000","last_undegraded":"2024-05-11T01:14:46.837590+0000","last_fullsized":"2024-05-11T01:14:46.837590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837594+0000","last_change":"2024-05-11T01:13:59.433723+0000","last_active":"2024-05-11T01:14:46.837594+0000","last_peered":"2024-05-11T01:14:46.837594+0000","last_clean":"2024-05-11T01:14:46.837594+0000","last_became_active":"2024-05-11T01:13:59.433441+0000","last_became_peered":"2024-05-11T01:13:59.433441+0000","last_unstale":"2024-05-11T01:14:46.837594+0000","last_undegraded":"2024-05-11T01:14:46.837594+0000","last_fullsized":"2024-05-11T01:14:46.837594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839249+0000","last_change":"2024-05-11T01:13:58.433974+0000","last_active":"2024-05-11T01:14:46.839249+0000","last_peered":"2024-05-11T01:14:46.839249+0000","last_clean":"2024-05-11T01:14:46.839249+0000","last_became_active":"2024-05-11T01:13:58.433649+0000","last_became_peered":"2024-05-11T01:13:58.433649+0000","last_unstale":"2024-05-11T01:14:46.839249+0000","last_undegraded":"2024-05-11T01:14:46.839249+0000","last_fullsized":"2024-05-11T01:14:46.839249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:59.322532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853811+0000","last_change":"2024-05-11T01:13:59.439364+0000","last_active":"2024-05-11T01:14:46.853811+0000","last_peered":"2024-05-11T01:14:46.853811+0000","last_clean":"2024-05-11T01:14:46.853811+0000","last_became_active":"2024-05-11T01:13:59.439073+0000","last_became_peered":"2024-05-11T01:13:59.439073+0000","last_unstale":"2024-05-11T01:14:46.853811+0000","last_undegraded":"2024-05-11T01:14:46.853811+0000","last_fullsized":"2024-05-11T01:14:46.853811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:48.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837019+0000","last_change":"2024-05-11T01:13:58.435449+0000","last_active":"2024-05-11T01:14:46.837019+0000","last_peered":"2024-05-11T01:14:46.837019+0000","last_clean":"2024-05-11T01:14:46.837019+0000","last_became_active":"2024-05-11T01:13:58.434345+0000","last_became_peered":"2024-05-11T01:13:58.434345+0000","last_unstale":"2024-05-11T01:14:46.837019+0000","last_undegraded":"2024-05-11T01:14:46.837019+0000","last_fullsized":"2024-05-11T01:14:46.837019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:08:58.718268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836974+0000","last_change":"2024-05-11T01:13:59.440315+0000","last_active":"2024-05-11T01:14:46.836974+0000","last_peered":"2024-05-11T01:14:46.836974+0000","last_clean":"2024-05-11T01:14:46.836974+0000","last_became_active":"2024-05-11T01:13:59.439153+0000","last_became_peered":"2024-05-11T01:13:59.439153+0000","last_unstale":"2024-05-11T01:14:46.836974+0000","last_undegraded":"2024-05-11T01:14:46.836974+0000","last_fullsized":"2024-05-11T01:14:46.836974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:38:04.924508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854374+0000","last_change":"2024-05-11T01:13:58.434357+0000","last_active":"2024-05-11T01:14:46.854374+0000","last_peered":"2024-05-11T01:14:46.854374+0000","last_clean":"2024-05-11T01:14:46.854374+0000","last_became_active":"2024-05-11T01:13:58.433392+0000","last_became_peered":"2024-05-11T01:13:58.433392+0000","last_unstale":"2024-05-11T01:14:46.854374+0000","last_undegraded":"2024-05-11T01:14:46.854374+0000","last_fullsized":"2024-05-11T01:14:46.854374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:14.276107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838856+0000","last_change":"2024-05-11T01:13:59.435145+0000","last_active":"2024-05-11T01:14:46.838856+0000","last_peered":"2024-05-11T01:14:46.838856+0000","last_clean":"2024-05-11T01:14:46.838856+0000","last_became_active":"2024-05-11T01:13:59.434862+0000","last_became_peered":"2024-05-11T01:13:59.434862+0000","last_unstale":"2024-05-11T01:14:46.838856+0000","last_undegraded":"2024-05-11T01:14:46.838856+0000","last_fullsized":"2024-05-11T01:14:46.838856+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:17.395853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837423+0000","last_change":"2024-05-11T01:13:58.439588+0000","last_active":"2024-05-11T01:14:46.837423+0000","last_peered":"2024-05-11T01:14:46.837423+0000","last_clean":"2024-05-11T01:14:46.837423+0000","last_became_active":"2024-05-11T01:13:58.439133+0000","last_became_peered":"2024-05-11T01:13:58.439133+0000","last_unstale":"2024-05-11T01:14:46.837423+0000","last_undegraded":"2024-05-11T01:14:46.837423+0000","last_fullsized":"2024-05-11T01:14:46.837423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854573+0000","last_change":"2024-05-11T01:13:59.448882+0000","last_active":"2024-05-11T01:14:46.854573+0000","last_peered":"2024-05-11T01:14:46.854573+0000","last_clean":"2024-05-11T01:14:46.854573+0000","last_became_active":"2024-05-11T01:13:59.448208+0000","last_became_peered":"2024-05-11T01:13:59.448208+0000","last_unstale":"2024-05-11T01:14:46.854573+0000","last_undegraded":"2024-05-11T01:14:46.854573+0000","last_fullsized":"2024-05-11T01:14:46.854573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839012+0000","last_change":"2024-05-11T01:13:58.428838+0000","last_active":"2024-05-11T01:14:46.839012+0000","last_peered":"2024-05-11T01:14:46.839012+0000","last_clean":"2024-05-11T01:14:46.839012+0000","last_became_active":"2024-05-11T01:13:58.428556+0000","last_became_peered":"2024-05-11T01:13:58.428556+0000","last_unstale":"2024-05-11T01:14:46.839012+0000","last_undegraded":"2024-05-11T01:14:46.839012+0000","last_fullsized":"2024-05-11T01:14:46.839012+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:01:35.971417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854781+0000","last_change":"2024-05-11T01:13:59.447010+0000","last_active":"2024-05-11T01:14:46.854781+0000","last_peered":"2024-05-11T01:14:46.854781+0000","last_clean":"2024-05-11T01:14:46.854781+0000","last_became_active":"2024-05-11T01:13:59.446713+0000","last_became_peered":"2024-05-11T01:13:59.446713+0000","last_unstale":"2024-05-11T01:14:46.854781+0000","last_undegraded":"2024-05-11T01:14:46.854781+0000","last_fullsized":"2024-05-11T01:14:46.854781+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:03:04.844818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853638+0000","last_change":"2024-05-11T01:13:58.428031+0000","last_active":"2024-05-11T01:14:46.853638+0000","last_peered":"2024-05-11T01:14:46.853638+0000","last_clean":"2024-05-11T01:14:46.853638+0000","last_became_active":"2024-05-11T01:13:58.427742+0000","last_became_peered":"2024-05-11T01:13:58.427742+0000","last_unstale":"2024-05-11T01:14:46.853638+0000","last_undegraded":"2024-05-11T01:14:46.853638+0000","last_fullsized":"2024-05-11T01:14:46.853638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836923+0000","last_change":"2024-05-11T01:13:59.431589+0000","last_active":"2024-05-11T01:14:46.836923+0000","last_peered":"2024-05-11T01:14:46.836923+0000","last_clean":"2024-05-11T01:14:46.836923+0000","last_became_active":"2024-05-11T01:13:59.431292+0000","last_became_peered":"2024-05-11T01:13:59.431292+0000","last_unstale":"2024-05-11T01:14:46.836923+0000","last_undegraded":"2024-05-11T01:14:46.836923+0000","last_fullsized":"2024-05-11T01:14:46.836923+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853944+0000","last_change":"2024-05-11T01:13:58.436246+0000","last_active":"2024-05-11T01:14:46.853944+0000","last_peered":"2024-05-11T01:14:46.853944+0000","last_clean":"2024-05-11T01:14:46.853944+0000","last_became_active":"2024-05-11T01:13:58.435967+0000","last_became_peered":"2024-05-11T01:13:58.435967+0000","last_unstale":"2024-05-11T01:14:46.853944+0000","last_undegraded":"2024-05-11T01:14:46.853944+0000","last_fullsized":"2024-05-11T01:14:46.853944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:22.242555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839175+0000","last_change":"2024-05-11T01:13:59.445506+0000","last_active":"2024-05-11T01:14:46.839175+0000","last_peered":"2024-05-11T01:14:46.839175+0000","last_clean":"2024-05-11T01:14:46.839175+0000","last_became_active":"2024-05-11T01:13:59.445232+0000","last_became_peered":"2024-05-11T01:13:59.445232+0000","last_unstale":"2024-05-11T01:14:46.839175+0000","last_undegraded":"2024-05-11T01:14:46.839175+0000","last_fullsized":"2024-05-11T01:14:46.839175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:18:27.371597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839300+0000","last_change":"2024-05-11T01:13:58.426187+0000","last_active":"2024-05-11T01:14:46.839300+0000","last_peered":"2024-05-11T01:14:46.839300+0000","last_clean":"2024-05-11T01:14:46.839300+0000","last_became_active":"2024-05-11T01:13:58.425913+0000","last_became_peered":"2024-05-11T01:13:58.425913+0000","last_unstale":"2024-05-11T01:14:46.839300+0000","last_undegraded":"2024-05-11T01:14:46.839300+0000","last_fullsized":"2024-05-11T01:14:46.839300+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854582+0000","last_change":"2024-05-11T01:13:59.431401+0000","last_active":"2024-05-11T01:14:46.854582+0000","last_peered":"2024-05-11T01:14:46.854582+0000","last_clean":"2024-05-11T01:14:46.854582+0000","last_became_active":"2024-05-11T01:13:59.431126+0000","last_became_peered":"2024-05-11T01:13:59.431126+0000","last_unstale":"2024-05-11T01:14:46.854582+0000","last_undegraded":"2024-05-11T01:14:46.854582+0000","last_fullsized":"2024-05-11T01:14:46.854582+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:42:47.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855202+0000","last_change":"2024-05-11T01:13:58.431417+0000","last_active":"2024-05-11T01:14:46.855202+0000","last_peered":"2024-05-11T01:14:46.855202+0000","last_clean":"2024-05-11T01:14:46.855202+0000","last_became_active":"2024-05-11T01:13:58.431132+0000","last_became_peered":"2024-05-11T01:13:58.431132+0000","last_unstale":"2024-05-11T01:14:46.855202+0000","last_undegraded":"2024-05-11T01:14:46.855202+0000","last_fullsized":"2024-05-11T01:14:46.855202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:23:16.977274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836973+0000","last_change":"2024-05-11T01:13:59.436396+0000","last_active":"2024-05-11T01:14:46.836973+0000","last_peered":"2024-05-11T01:14:46.836973+0000","last_clean":"2024-05-11T01:14:46.836973+0000","last_became_active":"2024-05-11T01:13:59.436043+0000","last_became_peered":"2024-05-11T01:13:59.436043+0000","last_unstale":"2024-05-11T01:14:46.836973+0000","last_undegraded":"2024-05-11T01:14:46.836973+0000","last_fullsized":"2024-05-11T01:14:46.836973+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854476+0000","last_change":"2024-05-11T01:13:58.435566+0000","last_active":"2024-05-11T01:14:46.854476+0000","last_peered":"2024-05-11T01:14:46.854476+0000","last_clean":"2024-05-11T01:14:46.854476+0000","last_became_active":"2024-05-11T01:13:58.435280+0000","last_became_peered":"2024-05-11T01:13:58.435280+0000","last_unstale":"2024-05-11T01:14:46.854476+0000","last_undegraded":"2024-05-11T01:14:46.854476+0000","last_fullsized":"2024-05-11T01:14:46.854476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:47.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854368+0000","last_change":"2024-05-11T01:13:59.432307+0000","last_active":"2024-05-11T01:14:46.854368+0000","last_peered":"2024-05-11T01:14:46.854368+0000","last_clean":"2024-05-11T01:14:46.854368+0000","last_became_active":"2024-05-11T01:13:59.431913+0000","last_became_peered":"2024-05-11T01:13:59.431913+0000","last_unstale":"2024-05-11T01:14:46.854368+0000","last_undegraded":"2024-05-11T01:14:46.854368+0000","last_fullsized":"2024-05-11T01:14:46.854368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"63'10","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:07.973661+0000","last_change":"2024-05-11T01:13:58.425157+0000","last_active":"2024-05-11T01:16:07.973661+0000","last_peered":"2024-05-11T01:16:07.973661+0000","last_clean":"2024-05-11T01:16:07.973661+0000","last_became_active":"2024-05-11T01:13:58.424894+0000","last_became_peered":"2024-05-11T01:13:58.424894+0000","last_unstale":"2024-05-11T01:16:07.973661+0000","last_undegraded":"2024-05-11T01:16:07.973661+0000","last_fullsized":"2024-05-11T01:16:07.973661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:48:11.424880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837159+0000","last_change":"2024-05-11T01:13:59.447362+0000","last_active":"2024-05-11T01:14:46.837159+0000","last_peered":"2024-05-11T01:14:46.837159+0000","last_clean":"2024-05-11T01:14:46.837159+0000","last_became_active":"2024-05-11T01:13:59.447082+0000","last_became_peered":"2024-05-11T01:13:59.447082+0000","last_unstale":"2024-05-11T01:14:46.837159+0000","last_undegraded":"2024-05-11T01:14:46.837159+0000","last_fullsized":"2024-05-11T01:14:46.837159+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837499+0000","last_change":"2024-05-11T01:13:58.439044+0000","last_active":"2024-05-11T01:14:46.837499+0000","last_peered":"2024-05-11T01:14:46.837499+0000","last_clean":"2024-05-11T01:14:46.837499+0000","last_became_active":"2024-05-11T01:13:58.438603+0000","last_became_peered":"2024-05-11T01:13:58.438603+0000","last_unstale":"2024-05-11T01:14:46.837499+0000","last_undegraded":"2024-05-11T01:14:46.837499+0000","last_fullsized":"2024-05-11T01:14:46.837499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854651+0000","last_change":"2024-05-11T01:13:59.449114+0000","last_active":"2024-05-11T01:14:46.854651+0000","last_peered":"2024-05-11T01:14:46.854651+0000","last_clean":"2024-05-11T01:14:46.854651+0000","last_became_active":"2024-05-11T01:13:59.448532+0000","last_became_peered":"2024-05-11T01:13:59.448532+0000","last_unstale":"2024-05-11T01:14:46.854651+0000","last_undegraded":"2024-05-11T01:14:46.854651+0000","last_fullsized":"2024-05-11T01:14:46.854651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854151+0000","last_change":"2024-05-11T01:13:58.434118+0000","last_active":"2024-05-11T01:14:46.854151+0000","last_peered":"2024-05-11T01:14:46.854151+0000","last_clean":"2024-05-11T01:14:46.854151+0000","last_became_active":"2024-05-11T01:13:58.433818+0000","last_became_peered":"2024-05-11T01:13:58.433818+0000","last_unstale":"2024-05-11T01:14:46.854151+0000","last_undegraded":"2024-05-11T01:14:46.854151+0000","last_fullsized":"2024-05-11T01:14:46.854151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:49.867304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836702+0000","last_change":"2024-05-11T01:13:59.435013+0000","last_active":"2024-05-11T01:14:46.836702+0000","last_peered":"2024-05-11T01:14:46.836702+0000","last_clean":"2024-05-11T01:14:46.836702+0000","last_became_active":"2024-05-11T01:13:59.434726+0000","last_became_peered":"2024-05-11T01:13:59.434726+0000","last_unstale":"2024-05-11T01:14:46.836702+0000","last_undegraded":"2024-05-11T01:14:46.836702+0000","last_fullsized":"2024-05-11T01:14:46.836702+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:03:22.763694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837125+0000","last_change":"2024-05-11T01:13:58.438894+0000","last_active":"2024-05-11T01:14:46.837125+0000","last_peered":"2024-05-11T01:14:46.837125+0000","last_clean":"2024-05-11T01:14:46.837125+0000","last_became_active":"2024-05-11T01:13:58.438528+0000","last_became_peered":"2024-05-11T01:13:58.438528+0000","last_unstale":"2024-05-11T01:14:46.837125+0000","last_undegraded":"2024-05-11T01:14:46.837125+0000","last_fullsized":"2024-05-11T01:14:46.837125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854117+0000","last_change":"2024-05-11T01:13:59.437777+0000","last_active":"2024-05-11T01:14:46.854117+0000","last_peered":"2024-05-11T01:14:46.854117+0000","last_clean":"2024-05-11T01:14:46.854117+0000","last_became_active":"2024-05-11T01:13:59.437176+0000","last_became_peered":"2024-05-11T01:13:59.437176+0000","last_unstale":"2024-05-11T01:14:46.854117+0000","last_undegraded":"2024-05-11T01:14:46.854117+0000","last_fullsized":"2024-05-11T01:14:46.854117+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:07:21.319542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854062+0000","last_change":"2024-05-11T01:13:58.431218+0000","last_active":"2024-05-11T01:14:46.854062+0000","last_peered":"2024-05-11T01:14:46.854062+0000","last_clean":"2024-05-11T01:14:46.854062+0000","last_became_active":"2024-05-11T01:13:58.430952+0000","last_became_peered":"2024-05-11T01:13:58.430952+0000","last_unstale":"2024-05-11T01:14:46.854062+0000","last_undegraded":"2024-05-11T01:14:46.854062+0000","last_fullsized":"2024-05-11T01:14:46.854062+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:03:39.335329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854664+0000","last_change":"2024-05-11T01:13:59.427029+0000","last_active":"2024-05-11T01:14:46.854664+0000","last_peered":"2024-05-11T01:14:46.854664+0000","last_clean":"2024-05-11T01:14:46.854664+0000","last_became_active":"2024-05-11T01:13:59.426727+0000","last_became_peered":"2024-05-11T01:13:59.426727+0000","last_unstale":"2024-05-11T01:14:46.854664+0000","last_undegraded":"2024-05-11T01:14:46.854664+0000","last_fullsized":"2024-05-11T01:14:46.854664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:52.626350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854202+0000","last_change":"2024-05-11T01:13:58.434507+0000","last_active":"2024-05-11T01:14:46.854202+0000","last_peered":"2024-05-11T01:14:46.854202+0000","last_clean":"2024-05-11T01:14:46.854202+0000","last_became_active":"2024-05-11T01:13:58.434215+0000","last_became_peered":"2024-05-11T01:13:58.434215+0000","last_unstale":"2024-05-11T01:14:46.854202+0000","last_undegraded":"2024-05-11T01:14:46.854202+0000","last_fullsized":"2024-05-11T01:14:46.854202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:35.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854789+0000","last_change":"2024-05-11T01:13:59.444388+0000","last_active":"2024-05-11T01:14:46.854789+0000","last_peered":"2024-05-11T01:14:46.854789+0000","last_clean":"2024-05-11T01:14:46.854789+0000","last_became_active":"2024-05-11T01:13:59.443981+0000","last_became_peered":"2024-05-11T01:13:59.443981+0000","last_unstale":"2024-05-11T01:14:46.854789+0000","last_undegraded":"2024-05-11T01:14:46.854789+0000","last_fullsized":"2024-05-11T01:14:46.854789+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839290+0000","last_change":"2024-05-11T01:13:58.429801+0000","last_active":"2024-05-11T01:14:46.839290+0000","last_peered":"2024-05-11T01:14:46.839290+0000","last_clean":"2024-05-11T01:14:46.839290+0000","last_became_active":"2024-05-11T01:13:58.429538+0000","last_became_peered":"2024-05-11T01:13:58.429538+0000","last_unstale":"2024-05-11T01:14:46.839290+0000","last_undegraded":"2024-05-11T01:14:46.839290+0000","last_fullsized":"2024-05-11T01:14:46.839290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:27.314762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837321+0000","last_change":"2024-05-11T01:13:59.438763+0000","last_active":"2024-05-11T01:14:46.837321+0000","last_peered":"2024-05-11T01:14:46.837321+0000","last_clean":"2024-05-11T01:14:46.837321+0000","last_became_active":"2024-05-11T01:13:59.438474+0000","last_became_peered":"2024-05-11T01:13:59.438474+0000","last_unstale":"2024-05-11T01:14:46.837321+0000","last_undegraded":"2024-05-11T01:14:46.837321+0000","last_fullsized":"2024-05-11T01:14:46.837321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854295+0000","last_change":"2024-05-11T01:13:58.426925+0000","last_active":"2024-05-11T01:14:46.854295+0000","last_peered":"2024-05-11T01:14:46.854295+0000","last_clean":"2024-05-11T01:14:46.854295+0000","last_became_active":"2024-05-11T01:13:58.426586+0000","last_became_peered":"2024-05-11T01:13:58.426586+0000","last_unstale":"2024-05-11T01:14:46.854295+0000","last_undegraded":"2024-05-11T01:14:46.854295+0000","last_fullsized":"2024-05-11T01:14:46.854295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837463+0000","last_change":"2024-05-11T01:13:59.436970+0000","last_active":"2024-05-11T01:14:46.837463+0000","last_peered":"2024-05-11T01:14:46.837463+0000","last_clean":"2024-05-11T01:14:46.837463+0000","last_became_active":"2024-05-11T01:13:59.436373+0000","last_became_peered":"2024-05-11T01:13:59.436373+0000","last_unstale":"2024-05-11T01:14:46.837463+0000","last_undegraded":"2024-05-11T01:14:46.837463+0000","last_fullsized":"2024-05-11T01:14:46.837463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837079+0000","last_change":"2024-05-11T01:13:58.439680+0000","last_active":"2024-05-11T01:14:46.837079+0000","last_peered":"2024-05-11T01:14:46.837079+0000","last_clean":"2024-05-11T01:14:46.837079+0000","last_became_active":"2024-05-11T01:13:58.439404+0000","last_became_peered":"2024-05-11T01:13:58.439404+0000","last_unstale":"2024-05-11T01:14:46.837079+0000","last_undegraded":"2024-05-11T01:14:46.837079+0000","last_fullsized":"2024-05-11T01:14:46.837079+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:41:26.236385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837268+0000","last_change":"2024-05-11T01:13:59.435753+0000","last_active":"2024-05-11T01:14:46.837268+0000","last_peered":"2024-05-11T01:14:46.837268+0000","last_clean":"2024-05-11T01:14:46.837268+0000","last_became_active":"2024-05-11T01:13:59.435253+0000","last_became_peered":"2024-05-11T01:13:59.435253+0000","last_unstale":"2024-05-11T01:14:46.837268+0000","last_undegraded":"2024-05-11T01:14:46.837268+0000","last_fullsized":"2024-05-11T01:14:46.837268+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:06:58.758258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837593+0000","last_change":"2024-05-11T01:13:58.437862+0000","last_active":"2024-05-11T01:14:46.837593+0000","last_peered":"2024-05-11T01:14:46.837593+0000","last_clean":"2024-05-11T01:14:46.837593+0000","last_became_active":"2024-05-11T01:13:58.437597+0000","last_became_peered":"2024-05-11T01:13:58.437597+0000","last_unstale":"2024-05-11T01:14:46.837593+0000","last_undegraded":"2024-05-11T01:14:46.837593+0000","last_fullsized":"2024-05-11T01:14:46.837593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:19:23.818943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836437+0000","last_change":"2024-05-11T01:13:59.431673+0000","last_active":"2024-05-11T01:14:46.836437+0000","last_peered":"2024-05-11T01:14:46.836437+0000","last_clean":"2024-05-11T01:14:46.836437+0000","last_became_active":"2024-05-11T01:13:59.431386+0000","last_became_peered":"2024-05-11T01:13:59.431386+0000","last_unstale":"2024-05-11T01:14:46.836437+0000","last_undegraded":"2024-05-11T01:14:46.836437+0000","last_fullsized":"2024-05-11T01:14:46.836437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:27.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853615+0000","last_change":"2024-05-11T01:13:58.431189+0000","last_active":"2024-05-11T01:14:46.853615+0000","last_peered":"2024-05-11T01:14:46.853615+0000","last_clean":"2024-05-11T01:14:46.853615+0000","last_became_active":"2024-05-11T01:13:58.430890+0000","last_became_peered":"2024-05-11T01:13:58.430890+0000","last_unstale":"2024-05-11T01:14:46.853615+0000","last_undegraded":"2024-05-11T01:14:46.853615+0000","last_fullsized":"2024-05-11T01:14:46.853615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854490+0000","last_change":"2024-05-11T01:13:59.429997+0000","last_active":"2024-05-11T01:14:46.854490+0000","last_peered":"2024-05-11T01:14:46.854490+0000","last_clean":"2024-05-11T01:14:46.854490+0000","last_became_active":"2024-05-11T01:13:59.429734+0000","last_became_peered":"2024-05-11T01:13:59.429734+0000","last_unstale":"2024-05-11T01:14:46.854490+0000","last_undegraded":"2024-05-11T01:14:46.854490+0000","last_fullsized":"2024-05-11T01:14:46.854490+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:47.445191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854269+0000","last_change":"2024-05-11T01:13:58.431136+0000","last_active":"2024-05-11T01:14:46.854269+0000","last_peered":"2024-05-11T01:14:46.854269+0000","last_clean":"2024-05-11T01:14:46.854269+0000","last_became_active":"2024-05-11T01:13:58.430846+0000","last_became_peered":"2024-05-11T01:13:58.430846+0000","last_unstale":"2024-05-11T01:14:46.854269+0000","last_undegraded":"2024-05-11T01:14:46.854269+0000","last_fullsized":"2024-05-11T01:14:46.854269+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:38.439492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854167+0000","last_change":"2024-05-11T01:13:59.450616+0000","last_active":"2024-05-11T01:14:46.854167+0000","last_peered":"2024-05-11T01:14:46.854167+0000","last_clean":"2024-05-11T01:14:46.854167+0000","last_became_active":"2024-05-11T01:13:59.450300+0000","last_became_peered":"2024-05-11T01:13:59.450300+0000","last_unstale":"2024-05-11T01:14:46.854167+0000","last_undegraded":"2024-05-11T01:14:46.854167+0000","last_fullsized":"2024-05-11T01:14:46.854167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854250+0000","last_change":"2024-05-11T01:13:58.436187+0000","last_active":"2024-05-11T01:14:46.854250+0000","last_peered":"2024-05-11T01:14:46.854250+0000","last_clean":"2024-05-11T01:14:46.854250+0000","last_became_active":"2024-05-11T01:13:58.434702+0000","last_became_peered":"2024-05-11T01:13:58.434702+0000","last_unstale":"2024-05-11T01:14:46.854250+0000","last_undegraded":"2024-05-11T01:14:46.854250+0000","last_fullsized":"2024-05-11T01:14:46.854250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:28.737362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837639+0000","last_change":"2024-05-11T01:13:59.452028+0000","last_active":"2024-05-11T01:14:46.837639+0000","last_peered":"2024-05-11T01:14:46.837639+0000","last_clean":"2024-05-11T01:14:46.837639+0000","last_became_active":"2024-05-11T01:13:59.451773+0000","last_became_peered":"2024-05-11T01:13:59.451773+0000","last_unstale":"2024-05-11T01:14:46.837639+0000","last_undegraded":"2024-05-11T01:14:46.837639+0000","last_fullsized":"2024-05-11T01:14:46.837639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"61'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837109+0000","last_change":"2024-05-11T01:13:58.436921+0000","last_active":"2024-05-11T01:14:46.837109+0000","last_peered":"2024-05-11T01:14:46.837109+0000","last_clean":"2024-05-11T01:14:46.837109+0000","last_became_active":"2024-05-11T01:13:58.436650+0000","last_became_peered":"2024-05-11T01:13:58.436650+0000","last_unstale":"2024-05-11T01:14:46.837109+0000","last_undegraded":"2024-05-11T01:14:46.837109+0000","last_fullsized":"2024-05-11T01:14:46.837109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:16:06.268571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854569+0000","last_change":"2024-05-11T01:13:59.442952+0000","last_active":"2024-05-11T01:14:46.854569+0000","last_peered":"2024-05-11T01:14:46.854569+0000","last_clean":"2024-05-11T01:14:46.854569+0000","last_became_active":"2024-05-11T01:13:59.442413+0000","last_became_peered":"2024-05-11T01:13:59.442413+0000","last_unstale":"2024-05-11T01:14:46.854569+0000","last_undegraded":"2024-05-11T01:14:46.854569+0000","last_fullsized":"2024-05-11T01:14:46.854569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:32:54.808774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854522+0000","last_change":"2024-05-11T01:13:58.436342+0000","last_active":"2024-05-11T01:14:46.854522+0000","last_peered":"2024-05-11T01:14:46.854522+0000","last_clean":"2024-05-11T01:14:46.854522+0000","last_became_active":"2024-05-11T01:13:58.436037+0000","last_became_peered":"2024-05-11T01:13:58.436037+0000","last_unstale":"2024-05-11T01:14:46.854522+0000","last_undegraded":"2024-05-11T01:14:46.854522+0000","last_fullsized":"2024-05-11T01:14:46.854522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:22.498662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854431+0000","last_change":"2024-05-11T01:13:59.434239+0000","last_active":"2024-05-11T01:14:46.854431+0000","last_peered":"2024-05-11T01:14:46.854431+0000","last_clean":"2024-05-11T01:14:46.854431+0000","last_became_active":"2024-05-11T01:13:59.433946+0000","last_became_peered":"2024-05-11T01:13:59.433946+0000","last_unstale":"2024-05-11T01:14:46.854431+0000","last_undegraded":"2024-05-11T01:14:46.854431+0000","last_fullsized":"2024-05-11T01:14:46.854431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:07:36.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837338+0000","last_change":"2024-05-11T01:13:58.432458+0000","last_active":"2024-05-11T01:14:46.837338+0000","last_peered":"2024-05-11T01:14:46.837338+0000","last_clean":"2024-05-11T01:14:46.837338+0000","last_became_active":"2024-05-11T01:13:58.432149+0000","last_became_peered":"2024-05-11T01:13:58.432149+0000","last_unstale":"2024-05-11T01:14:46.837338+0000","last_undegraded":"2024-05-11T01:14:46.837338+0000","last_fullsized":"2024-05-11T01:14:46.837338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837389+0000","last_change":"2024-05-11T01:13:59.432345+0000","last_active":"2024-05-11T01:14:46.837389+0000","last_peered":"2024-05-11T01:14:46.837389+0000","last_clean":"2024-05-11T01:14:46.837389+0000","last_became_active":"2024-05-11T01:13:59.432058+0000","last_became_peered":"2024-05-11T01:13:59.432058+0000","last_unstale":"2024-05-11T01:14:46.837389+0000","last_undegraded":"2024-05-11T01:14:46.837389+0000","last_fullsized":"2024-05-11T01:14:46.837389+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:38:32.025100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854602+0000","last_change":"2024-05-11T01:13:58.433877+0000","last_active":"2024-05-11T01:14:46.854602+0000","last_peered":"2024-05-11T01:14:46.854602+0000","last_clean":"2024-05-11T01:14:46.854602+0000","last_became_active":"2024-05-11T01:13:58.433584+0000","last_became_peered":"2024-05-11T01:13:58.433584+0000","last_unstale":"2024-05-11T01:14:46.854602+0000","last_undegraded":"2024-05-11T01:14:46.854602+0000","last_fullsized":"2024-05-11T01:14:46.854602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:20.805314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837457+0000","last_change":"2024-05-11T01:13:59.434011+0000","last_active":"2024-05-11T01:14:46.837457+0000","last_peered":"2024-05-11T01:14:46.837457+0000","last_clean":"2024-05-11T01:14:46.837457+0000","last_became_active":"2024-05-11T01:13:59.432337+0000","last_became_peered":"2024-05-11T01:13:59.432337+0000","last_unstale":"2024-05-11T01:14:46.837457+0000","last_undegraded":"2024-05-11T01:14:46.837457+0000","last_fullsized":"2024-05-11T01:14:46.837457+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839100+0000","last_change":"2024-05-11T01:13:58.435211+0000","last_active":"2024-05-11T01:14:46.839100+0000","last_peered":"2024-05-11T01:14:46.839100+0000","last_clean":"2024-05-11T01:14:46.839100+0000","last_became_active":"2024-05-11T01:13:58.434956+0000","last_became_peered":"2024-05-11T01:13:58.434956+0000","last_unstale":"2024-05-11T01:14:46.839100+0000","last_undegraded":"2024-05-11T01:14:46.839100+0000","last_fullsized":"2024-05-11T01:14:46.839100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853560+0000","last_change":"2024-05-11T01:13:59.432319+0000","last_active":"2024-05-11T01:14:46.853560+0000","last_peered":"2024-05-11T01:14:46.853560+0000","last_clean":"2024-05-11T01:14:46.853560+0000","last_became_active":"2024-05-11T01:13:59.432012+0000","last_became_peered":"2024-05-11T01:13:59.432012+0000","last_unstale":"2024-05-11T01:14:46.853560+0000","last_undegraded":"2024-05-11T01:14:46.853560+0000","last_fullsized":"2024-05-11T01:14:46.853560+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:21.807568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836595+0000","last_change":"2024-05-11T01:13:58.433752+0000","last_active":"2024-05-11T01:14:46.836595+0000","last_peered":"2024-05-11T01:14:46.836595+0000","last_clean":"2024-05-11T01:14:46.836595+0000","last_became_active":"2024-05-11T01:13:58.433411+0000","last_became_peered":"2024-05-11T01:13:58.433411+0000","last_unstale":"2024-05-11T01:14:46.836595+0000","last_undegraded":"2024-05-11T01:14:46.836595+0000","last_fullsized":"2024-05-11T01:14:46.836595+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838906+0000","last_change":"2024-05-11T01:13:59.446971+0000","last_active":"2024-05-11T01:14:46.838906+0000","last_peered":"2024-05-11T01:14:46.838906+0000","last_clean":"2024-05-11T01:14:46.838906+0000","last_became_active":"2024-05-11T01:13:59.446670+0000","last_became_peered":"2024-05-11T01:13:59.446670+0000","last_unstale":"2024-05-11T01:14:46.838906+0000","last_undegraded":"2024-05-11T01:14:46.838906+0000","last_fullsized":"2024-05-11T01:14:46.838906+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:45:46.843918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854241+0000","last_change":"2024-05-11T01:13:58.435876+0000","last_active":"2024-05-11T01:14:46.854241+0000","last_peered":"2024-05-11T01:14:46.854241+0000","last_clean":"2024-05-11T01:14:46.854241+0000","last_became_active":"2024-05-11T01:13:58.434062+0000","last_became_peered":"2024-05-11T01:13:58.434062+0000","last_unstale":"2024-05-11T01:14:46.854241+0000","last_undegraded":"2024-05-11T01:14:46.854241+0000","last_fullsized":"2024-05-11T01:14:46.854241+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:12:26.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855069+0000","last_change":"2024-05-11T01:13:59.445426+0000","last_active":"2024-05-11T01:14:46.855069+0000","last_peered":"2024-05-11T01:14:46.855069+0000","last_clean":"2024-05-11T01:14:46.855069+0000","last_became_active":"2024-05-11T01:13:59.445146+0000","last_became_peered":"2024-05-11T01:13:59.445146+0000","last_unstale":"2024-05-11T01:14:46.855069+0000","last_undegraded":"2024-05-11T01:14:46.855069+0000","last_fullsized":"2024-05-11T01:14:46.855069+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:57:36.879340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854223+0000","last_change":"2024-05-11T01:13:58.434600+0000","last_active":"2024-05-11T01:14:46.854223+0000","last_peered":"2024-05-11T01:14:46.854223+0000","last_clean":"2024-05-11T01:14:46.854223+0000","last_became_active":"2024-05-11T01:13:58.434320+0000","last_became_peered":"2024-05-11T01:13:58.434320+0000","last_unstale":"2024-05-11T01:14:46.854223+0000","last_undegraded":"2024-05-11T01:14:46.854223+0000","last_fullsized":"2024-05-11T01:14:46.854223+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"51'86","reported_seq":246,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853980+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:14:46.853980+0000","last_peered":"2024-05-11T01:14:46.853980+0000","last_clean":"2024-05-11T01:14:46.853980+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:14:46.853980+0000","last_undegraded":"2024-05-11T01:14:46.853980+0000","last_fullsized":"2024-05-11T01:14:46.853980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3807,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":949409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542331,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":757760,"data_stored":359623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54128,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317712,"statfs":{"total":96636764160,"available":96581337088,"internally_reserved":0,"allocated":761856,"data_stored":361960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575033,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44408,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327432,"statfs":{"total":96636764160,"available":96591290368,"internally_reserved":0,"allocated":761856,"data_stored":359589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":765952,"data_stored":361982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58288,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313552,"statfs":{"total":96636764160,"available":96577077248,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575033,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1347584,"data_stored":949353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:16:15.172 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:16:15.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.312+0000 7fdafd46d640 1 Processor -- start 2024-05-11T01:16:15.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.312+0000 7fdafd46d640 1 -- start start 2024-05-11T01:16:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdafd46d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdaf8072ce0 0x7fdaf806e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdafd46d640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdaf80afb30 0x7fdaf80730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdafd46d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf80aea90 con 0x7fdaf8073600 2024-05-11T01:16:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdafd46d640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf80b59e0 con 0x7fdaf80afb30 2024-05-11T01:16:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdafd46d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf80ae860 con 0x7fdaf8072ce0 2024-05-11T01:16:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdaf67fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdaf80afb30 0x7fdaf80730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdaf67fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdaf80afb30 0x7fdaf80730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43348/0 (socket says 172.21.15.52:43348) 2024-05-11T01:16:15.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdaf67fc640 1 -- 172.21.15.52:0/2196553151 learned_addr learned my addr 172.21.15.52:0/2196553151 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdaf6ffd640 1 --2- 172.21.15.52:0/2196553151 >> v2:172.21.15.119:3300/0 conn(0x7fdaf8072ce0 0x7fdaf806e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdaf67fc640 1 -- 172.21.15.52:0/2196553151 >> v2:172.21.15.119:3300/0 conn(0x7fdaf8072ce0 msgr2=0x7fdaf806e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdaf67fc640 1 --2- 172.21.15.52:0/2196553151 >> v2:172.21.15.119:3300/0 conn(0x7fdaf8072ce0 0x7fdaf806e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdaf67fc640 1 -- 172.21.15.52:0/2196553151 >> v1:172.21.15.119:6789/0 conn(0x7fdaf8073600 legacy=0x7fdaf80b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.313+0000 7fdaf67fc640 1 -- 172.21.15.52:0/2196553151 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf80aed70 con 0x7fdaf80afb30 2024-05-11T01:16:15.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.314+0000 7fdaf67fc640 1 --2- 172.21.15.52:0/2196553151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf80afb30 0x7fdaf80730c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80b3ac0 tx=0x7fdaec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7326c3621f5a24d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:15.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.314+0000 7fdaf5ffb640 1 -- 172.21.15.52:0/2196553151 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec037070 con 0x7fdaf80afb30 2024-05-11T01:16:15.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.314+0000 7fdaf5ffb640 1 -- 172.21.15.52:0/2196553151 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec032440 con 0x7fdaf80afb30 2024-05-11T01:16:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.314+0000 7fdaf5ffb640 1 -- 172.21.15.52:0/2196553151 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec032810 con 0x7fdaf80afb30 2024-05-11T01:16:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.315+0000 7fdafd46d640 1 -- 172.21.15.52:0/2196553151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf80afb30 msgr2=0x7fdaf80730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.315+0000 7fdafd46d640 1 --2- 172.21.15.52:0/2196553151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf80afb30 0x7fdaf80730c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80b3ac0 tx=0x7fdaec02a1e0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.315+0000 7fdafd46d640 1 -- 172.21.15.52:0/2196553151 shutdown_connections 2024-05-11T01:16:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.315+0000 7fdafd46d640 1 --2- 172.21.15.52:0/2196553151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf80afb30 0x7fdaf80730c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.315+0000 7fdafd46d640 1 --2- 172.21.15.52:0/2196553151 >> v2:172.21.15.119:3300/0 conn(0x7fdaf8072ce0 0x7fdaf806e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.315+0000 7fdafd46d640 1 -- 172.21.15.52:0/2196553151 >> 172.21.15.52:0/2196553151 conn(0x7fdaf80b2a00 msgr2=0x7fdaf80b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:15.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.315+0000 7fdafd46d640 1 -- 172.21.15.52:0/2196553151 shutdown_connections 2024-05-11T01:16:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.315+0000 7fdafd46d640 1 -- 172.21.15.52:0/2196553151 wait complete. 2024-05-11T01:16:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.316+0000 7fdafd46d640 1 Processor -- start 2024-05-11T01:16:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.316+0000 7fdafd46d640 1 -- start start 2024-05-11T01:16:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdafd46d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf8072ce0 0x7fdaf81e9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdafd46d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf8073600 0x7fdaf81e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdafd46d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdaf80afb30 0x7fdaf8145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdafd46d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf80af610 con 0x7fdaf8073600 2024-05-11T01:16:15.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdafd46d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf80af330 con 0x7fdaf8072ce0 2024-05-11T01:16:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdafd46d640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf8073a30 con 0x7fdaf80afb30 2024-05-11T01:16:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdaf67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf8073600 0x7fdaf81e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdaf67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf8073600 0x7fdaf81e4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43362/0 (socket says 172.21.15.52:43362) 2024-05-11T01:16:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdaf67fc640 1 -- 172.21.15.52:0/242782703 learned_addr learned my addr 172.21.15.52:0/242782703 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdaf67fc640 1 -- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdaf80afb30 msgr2=0x7fdaf8145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdaf67fc640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdaf80afb30 0x7fdaf8145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdaf6ffd640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf8072ce0 0x7fdaf81e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdaf67fc640 1 -- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf8072ce0 msgr2=0x7fdaf81e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdaf67fc640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf8072ce0 0x7fdaf81e9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.317+0000 7fdaf67fc640 1 -- 172.21.15.52:0/242782703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81e5720 con 0x7fdaf8073600 2024-05-11T01:16:15.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.318+0000 7fdaf67fc640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf8073600 0x7fdaf81e4ef0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000c00 tx=0x7fdaec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:15.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.318+0000 7fdadf7fe640 1 -- 172.21.15.52:0/242782703 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec041020 con 0x7fdaf8073600 2024-05-11T01:16:15.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.318+0000 7fdadf7fe640 1 -- 172.21.15.52:0/242782703 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec0306c0 con 0x7fdaf8073600 2024-05-11T01:16:15.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.318+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf8146870 con 0x7fdaf8073600 2024-05-11T01:16:15.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.318+0000 7fdadf7fe640 1 -- 172.21.15.52:0/242782703 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec037040 con 0x7fdaf8073600 2024-05-11T01:16:15.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.318+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf8146e30 con 0x7fdaf8073600 2024-05-11T01:16:15.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.319+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf8069780 con 0x7fdaf8073600 2024-05-11T01:16:15.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.320+0000 7fdadf7fe640 1 -- 172.21.15.52:0/242782703 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fdaec044020 con 0x7fdaf8073600 2024-05-11T01:16:15.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.320+0000 7fdadf7fe640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdae4076300 0x7fdae40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.320+0000 7fdadf7fe640 1 -- 172.21.15.52:0/242782703 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7fdaec0b80f0 con 0x7fdaf8073600 2024-05-11T01:16:15.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.321+0000 7fdaf6ffd640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdae4076300 0x7fdae40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.321+0000 7fdaf6ffd640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdae4076300 0x7fdae40787c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009f30 tx=0x7fdae800b040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:15.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.325+0000 7fdadf7fe640 1 -- 172.21.15.52:0/242782703 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdaec04b030 con 0x7fdaf8073600 2024-05-11T01:16:15.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.515+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdaf80b0470 con 0x7fdae4076300 2024-05-11T01:16:15.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.521+0000 7fdadf7fe640 1 -- 172.21.15.52:0/242782703 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251840 (secure 0 0 0) 0x7fdaf80b0470 con 0x7fdae4076300 2024-05-11T01:16:15.524 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:16:15.525 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:16:15.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdae4076300 msgr2=0x7fdae40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdae4076300 0x7fdae40787c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009f30 tx=0x7fdae800b040 comp rx=0 tx=0).stop 2024-05-11T01:16:15.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf8073600 msgr2=0x7fdaf81e4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf8073600 0x7fdaf81e4ef0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000c00 tx=0x7fdaec0046c0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 shutdown_connections 2024-05-11T01:16:15.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdae4076300 0x7fdae40787c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdaf80afb30 0x7fdaf8145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf8073600 0x7fdaf81e4ef0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 --2- 172.21.15.52:0/242782703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf8072ce0 0x7fdaf81e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 >> 172.21.15.52:0/242782703 conn(0x7fdaf80b2a00 msgr2=0x7fdaf80b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:15.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 shutdown_connections 2024-05-11T01:16:15.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.524+0000 7fdafd46d640 1 -- 172.21.15.52:0/242782703 wait complete. 2024-05-11T01:16:15.535 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-05-11T01:16:13.739942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594175,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":174,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317588,"kb_used_data":7080,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754657132,"statfs":{"total":773094113280,"available":772768903168,"internally_reserved":0,"allocated":7249920,"data_stored":4059958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":317836849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003405"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854522+0000","last_change":"2024-05-11T01:13:59.434893+0000","last_active":"2024-05-11T01:14:46.854522+0000","last_peered":"2024-05-11T01:14:46.854522+0000","last_clean":"2024-05-11T01:14:46.854522+0000","last_became_active":"2024-05-11T01:13:59.434594+0000","last_became_peered":"2024-05-11T01:13:59.434594+0000","last_unstale":"2024-05-11T01:14:46.854522+0000","last_undegraded":"2024-05-11T01:14:46.854522+0000","last_fullsized":"2024-05-11T01:14:46.854522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:49:33.685964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854143+0000","last_change":"2024-05-11T01:13:59.446282+0000","last_active":"2024-05-11T01:14:46.854143+0000","last_peered":"2024-05-11T01:14:46.854143+0000","last_clean":"2024-05-11T01:14:46.854143+0000","last_became_active":"2024-05-11T01:13:59.445971+0000","last_became_peered":"2024-05-11T01:13:59.445971+0000","last_unstale":"2024-05-11T01:14:46.854143+0000","last_undegraded":"2024-05-11T01:14:46.854143+0000","last_fullsized":"2024-05-11T01:14:46.854143+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:50.869559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839269+0000","last_change":"2024-05-11T01:13:59.430661+0000","last_active":"2024-05-11T01:14:46.839269+0000","last_peered":"2024-05-11T01:14:46.839269+0000","last_clean":"2024-05-11T01:14:46.839269+0000","last_became_active":"2024-05-11T01:13:59.430408+0000","last_became_peered":"2024-05-11T01:13:59.430408+0000","last_unstale":"2024-05-11T01:14:46.839269+0000","last_undegraded":"2024-05-11T01:14:46.839269+0000","last_fullsized":"2024-05-11T01:14:46.839269+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:14:03.379607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854537+0000","last_change":"2024-05-11T01:13:59.448424+0000","last_active":"2024-05-11T01:14:46.854537+0000","last_peered":"2024-05-11T01:14:46.854537+0000","last_clean":"2024-05-11T01:14:46.854537+0000","last_became_active":"2024-05-11T01:13:59.446986+0000","last_became_peered":"2024-05-11T01:13:59.446986+0000","last_unstale":"2024-05-11T01:14:46.854537+0000","last_undegraded":"2024-05-11T01:14:46.854537+0000","last_fullsized":"2024-05-11T01:14:46.854537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853610+0000","last_change":"2024-05-11T01:13:59.446369+0000","last_active":"2024-05-11T01:14:46.853610+0000","last_peered":"2024-05-11T01:14:46.853610+0000","last_clean":"2024-05-11T01:14:46.853610+0000","last_became_active":"2024-05-11T01:13:59.445922+0000","last_became_peered":"2024-05-11T01:13:59.445922+0000","last_unstale":"2024-05-11T01:14:46.853610+0000","last_undegraded":"2024-05-11T01:14:46.853610+0000","last_fullsized":"2024-05-11T01:14:46.853610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:15.278361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837202+0000","last_change":"2024-05-11T01:13:59.435535+0000","last_active":"2024-05-11T01:14:46.837202+0000","last_peered":"2024-05-11T01:14:46.837202+0000","last_clean":"2024-05-11T01:14:46.837202+0000","last_became_active":"2024-05-11T01:13:59.434634+0000","last_became_peered":"2024-05-11T01:13:59.434634+0000","last_unstale":"2024-05-11T01:14:46.837202+0000","last_undegraded":"2024-05-11T01:14:46.837202+0000","last_fullsized":"2024-05-11T01:14:46.837202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:34:16.451030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854001+0000","last_change":"2024-05-11T01:13:59.434552+0000","last_active":"2024-05-11T01:14:46.854001+0000","last_peered":"2024-05-11T01:14:46.854001+0000","last_clean":"2024-05-11T01:14:46.854001+0000","last_became_active":"2024-05-11T01:13:59.433976+0000","last_became_peered":"2024-05-11T01:13:59.433976+0000","last_unstale":"2024-05-11T01:14:46.854001+0000","last_undegraded":"2024-05-11T01:14:46.854001+0000","last_fullsized":"2024-05-11T01:14:46.854001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853803+0000","last_change":"2024-05-11T01:13:59.447658+0000","last_active":"2024-05-11T01:14:46.853803+0000","last_peered":"2024-05-11T01:14:46.853803+0000","last_clean":"2024-05-11T01:14:46.853803+0000","last_became_active":"2024-05-11T01:13:59.446507+0000","last_became_peered":"2024-05-11T01:13:59.446507+0000","last_unstale":"2024-05-11T01:14:46.853803+0000","last_undegraded":"2024-05-11T01:14:46.853803+0000","last_fullsized":"2024-05-11T01:14:46.853803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:34:51.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854683+0000","last_change":"2024-05-11T01:13:59.449973+0000","last_active":"2024-05-11T01:14:46.854683+0000","last_peered":"2024-05-11T01:14:46.854683+0000","last_clean":"2024-05-11T01:14:46.854683+0000","last_became_active":"2024-05-11T01:13:59.449670+0000","last_became_peered":"2024-05-11T01:13:59.449670+0000","last_unstale":"2024-05-11T01:14:46.854683+0000","last_undegraded":"2024-05-11T01:14:46.854683+0000","last_fullsized":"2024-05-11T01:14:46.854683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854986+0000","last_change":"2024-05-11T01:13:59.450134+0000","last_active":"2024-05-11T01:14:46.854986+0000","last_peered":"2024-05-11T01:14:46.854986+0000","last_clean":"2024-05-11T01:14:46.854986+0000","last_became_active":"2024-05-11T01:13:59.449822+0000","last_became_peered":"2024-05-11T01:13:59.449822+0000","last_unstale":"2024-05-11T01:14:46.854986+0000","last_undegraded":"2024-05-11T01:14:46.854986+0000","last_fullsized":"2024-05-11T01:14:46.854986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.840088+0000","last_change":"2024-05-11T01:14:46.840088+0000","last_active":"2024-05-11T01:14:46.840088+0000","last_peered":"2024-05-11T01:14:46.840088+0000","last_clean":"2024-05-11T01:14:46.840088+0000","last_became_active":"2024-05-11T01:14:46.839837+0000","last_became_peered":"2024-05-11T01:14:46.839837+0000","last_unstale":"2024-05-11T01:14:46.840088+0000","last_undegraded":"2024-05-11T01:14:46.840088+0000","last_fullsized":"2024-05-11T01:14:46.840088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:38:11.978713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853737+0000","last_change":"2024-05-11T01:13:59.428513+0000","last_active":"2024-05-11T01:14:46.853737+0000","last_peered":"2024-05-11T01:14:46.853737+0000","last_clean":"2024-05-11T01:14:46.853737+0000","last_became_active":"2024-05-11T01:13:59.428202+0000","last_became_peered":"2024-05-11T01:13:59.428202+0000","last_unstale":"2024-05-11T01:14:46.853737+0000","last_undegraded":"2024-05-11T01:14:46.853737+0000","last_fullsized":"2024-05-11T01:14:46.853737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:23.500916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854223+0000","last_change":"2024-05-11T01:13:59.435402+0000","last_active":"2024-05-11T01:14:46.854223+0000","last_peered":"2024-05-11T01:14:46.854223+0000","last_clean":"2024-05-11T01:14:46.854223+0000","last_became_active":"2024-05-11T01:13:59.433324+0000","last_became_peered":"2024-05-11T01:13:59.433324+0000","last_unstale":"2024-05-11T01:14:46.854223+0000","last_undegraded":"2024-05-11T01:14:46.854223+0000","last_fullsized":"2024-05-11T01:14:46.854223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:36:46.612605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854315+0000","last_change":"2024-05-11T01:13:59.447113+0000","last_active":"2024-05-11T01:14:46.854315+0000","last_peered":"2024-05-11T01:14:46.854315+0000","last_clean":"2024-05-11T01:14:46.854315+0000","last_became_active":"2024-05-11T01:13:59.446349+0000","last_became_peered":"2024-05-11T01:13:59.446349+0000","last_unstale":"2024-05-11T01:14:46.854315+0000","last_undegraded":"2024-05-11T01:14:46.854315+0000","last_fullsized":"2024-05-11T01:14:46.854315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836869+0000","last_change":"2024-05-11T01:13:59.437811+0000","last_active":"2024-05-11T01:14:46.836869+0000","last_peered":"2024-05-11T01:14:46.836869+0000","last_clean":"2024-05-11T01:14:46.836869+0000","last_became_active":"2024-05-11T01:13:59.427975+0000","last_became_peered":"2024-05-11T01:13:59.427975+0000","last_unstale":"2024-05-11T01:14:46.836869+0000","last_undegraded":"2024-05-11T01:14:46.836869+0000","last_fullsized":"2024-05-11T01:14:46.836869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:05:27.108182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836920+0000","last_change":"2024-05-11T01:13:59.446666+0000","last_active":"2024-05-11T01:14:46.836920+0000","last_peered":"2024-05-11T01:14:46.836920+0000","last_clean":"2024-05-11T01:14:46.836920+0000","last_became_active":"2024-05-11T01:13:59.446396+0000","last_became_peered":"2024-05-11T01:13:59.446396+0000","last_unstale":"2024-05-11T01:14:46.836920+0000","last_undegraded":"2024-05-11T01:14:46.836920+0000","last_fullsized":"2024-05-11T01:14:46.836920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:55:23.003515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839156+0000","last_change":"2024-05-11T01:13:59.437261+0000","last_active":"2024-05-11T01:14:46.839156+0000","last_peered":"2024-05-11T01:14:46.839156+0000","last_clean":"2024-05-11T01:14:46.839156+0000","last_became_active":"2024-05-11T01:13:59.436992+0000","last_became_peered":"2024-05-11T01:13:59.436992+0000","last_unstale":"2024-05-11T01:14:46.839156+0000","last_undegraded":"2024-05-11T01:14:46.839156+0000","last_fullsized":"2024-05-11T01:14:46.839156+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:53.116569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839026+0000","last_change":"2024-05-11T01:13:59.445508+0000","last_active":"2024-05-11T01:14:46.839026+0000","last_peered":"2024-05-11T01:14:46.839026+0000","last_clean":"2024-05-11T01:14:46.839026+0000","last_became_active":"2024-05-11T01:13:59.445318+0000","last_became_peered":"2024-05-11T01:13:59.445318+0000","last_unstale":"2024-05-11T01:14:46.839026+0000","last_undegraded":"2024-05-11T01:14:46.839026+0000","last_fullsized":"2024-05-11T01:14:46.839026+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:21.154101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855679+0000","last_change":"2024-05-11T01:14:46.855679+0000","last_active":"2024-05-11T01:14:46.855679+0000","last_peered":"2024-05-11T01:14:46.855679+0000","last_clean":"2024-05-11T01:14:46.855679+0000","last_became_active":"2024-05-11T01:14:46.855320+0000","last_became_peered":"2024-05-11T01:14:46.855320+0000","last_unstale":"2024-05-11T01:14:46.855679+0000","last_undegraded":"2024-05-11T01:14:46.855679+0000","last_fullsized":"2024-05-11T01:14:46.855679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:48:36.304487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854538+0000","last_change":"2024-05-11T01:13:59.444231+0000","last_active":"2024-05-11T01:14:46.854538+0000","last_peered":"2024-05-11T01:14:46.854538+0000","last_clean":"2024-05-11T01:14:46.854538+0000","last_became_active":"2024-05-11T01:13:59.443762+0000","last_became_peered":"2024-05-11T01:13:59.443762+0000","last_unstale":"2024-05-11T01:14:46.854538+0000","last_undegraded":"2024-05-11T01:14:46.854538+0000","last_fullsized":"2024-05-11T01:14:46.854538+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:25:17.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837269+0000","last_change":"2024-05-11T01:13:59.437781+0000","last_active":"2024-05-11T01:14:46.837269+0000","last_peered":"2024-05-11T01:14:46.837269+0000","last_clean":"2024-05-11T01:14:46.837269+0000","last_became_active":"2024-05-11T01:13:59.437444+0000","last_became_peered":"2024-05-11T01:13:59.437444+0000","last_unstale":"2024-05-11T01:14:46.837269+0000","last_undegraded":"2024-05-11T01:14:46.837269+0000","last_fullsized":"2024-05-11T01:14:46.837269+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:31:12.326102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854442+0000","last_change":"2024-05-11T01:13:59.447343+0000","last_active":"2024-05-11T01:14:46.854442+0000","last_peered":"2024-05-11T01:14:46.854442+0000","last_clean":"2024-05-11T01:14:46.854442+0000","last_became_active":"2024-05-11T01:13:59.446779+0000","last_became_peered":"2024-05-11T01:13:59.446779+0000","last_unstale":"2024-05-11T01:14:46.854442+0000","last_undegraded":"2024-05-11T01:14:46.854442+0000","last_fullsized":"2024-05-11T01:14:46.854442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836924+0000","last_change":"2024-05-11T01:13:59.431582+0000","last_active":"2024-05-11T01:14:46.836924+0000","last_peered":"2024-05-11T01:14:46.836924+0000","last_clean":"2024-05-11T01:14:46.836924+0000","last_became_active":"2024-05-11T01:13:59.431309+0000","last_became_peered":"2024-05-11T01:13:59.431309+0000","last_unstale":"2024-05-11T01:14:46.836924+0000","last_undegraded":"2024-05-11T01:14:46.836924+0000","last_fullsized":"2024-05-11T01:14:46.836924+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837032+0000","last_change":"2024-05-11T01:13:59.433902+0000","last_active":"2024-05-11T01:14:46.837032+0000","last_peered":"2024-05-11T01:14:46.837032+0000","last_clean":"2024-05-11T01:14:46.837032+0000","last_became_active":"2024-05-11T01:13:59.433606+0000","last_became_peered":"2024-05-11T01:13:59.433606+0000","last_unstale":"2024-05-11T01:14:46.837032+0000","last_undegraded":"2024-05-11T01:14:46.837032+0000","last_fullsized":"2024-05-11T01:14:46.837032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837537+0000","last_change":"2024-05-11T01:13:59.436026+0000","last_active":"2024-05-11T01:14:46.837537+0000","last_peered":"2024-05-11T01:14:46.837537+0000","last_clean":"2024-05-11T01:14:46.837537+0000","last_became_active":"2024-05-11T01:13:59.435708+0000","last_became_peered":"2024-05-11T01:13:59.435708+0000","last_unstale":"2024-05-11T01:14:46.837537+0000","last_undegraded":"2024-05-11T01:14:46.837537+0000","last_fullsized":"2024-05-11T01:14:46.837537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854605+0000","last_change":"2024-05-11T01:13:59.441132+0000","last_active":"2024-05-11T01:14:46.854605+0000","last_peered":"2024-05-11T01:14:46.854605+0000","last_clean":"2024-05-11T01:14:46.854605+0000","last_became_active":"2024-05-11T01:13:59.440852+0000","last_became_peered":"2024-05-11T01:13:59.440852+0000","last_unstale":"2024-05-11T01:14:46.854605+0000","last_undegraded":"2024-05-11T01:14:46.854605+0000","last_fullsized":"2024-05-11T01:14:46.854605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:18:58.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839326+0000","last_change":"2024-05-11T01:13:59.436547+0000","last_active":"2024-05-11T01:14:46.839326+0000","last_peered":"2024-05-11T01:14:46.839326+0000","last_clean":"2024-05-11T01:14:46.839326+0000","last_became_active":"2024-05-11T01:13:59.436260+0000","last_became_peered":"2024-05-11T01:13:59.436260+0000","last_unstale":"2024-05-11T01:14:46.839326+0000","last_undegraded":"2024-05-11T01:14:46.839326+0000","last_fullsized":"2024-05-11T01:14:46.839326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:27.176219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853867+0000","last_change":"2024-05-11T01:13:59.425177+0000","last_active":"2024-05-11T01:14:46.853867+0000","last_peered":"2024-05-11T01:14:46.853867+0000","last_clean":"2024-05-11T01:14:46.853867+0000","last_became_active":"2024-05-11T01:13:59.424328+0000","last_became_peered":"2024-05-11T01:13:59.424328+0000","last_unstale":"2024-05-11T01:14:46.853867+0000","last_undegraded":"2024-05-11T01:14:46.853867+0000","last_fullsized":"2024-05-11T01:14:46.853867+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:29.739616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837190+0000","last_change":"2024-05-11T01:13:59.436551+0000","last_active":"2024-05-11T01:14:46.837190+0000","last_peered":"2024-05-11T01:14:46.837190+0000","last_clean":"2024-05-11T01:14:46.837190+0000","last_became_active":"2024-05-11T01:13:59.436257+0000","last_became_peered":"2024-05-11T01:13:59.436257+0000","last_unstale":"2024-05-11T01:14:46.837190+0000","last_undegraded":"2024-05-11T01:14:46.837190+0000","last_fullsized":"2024-05-11T01:14:46.837190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:37:44.925797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838948+0000","last_change":"2024-05-11T01:13:59.435398+0000","last_active":"2024-05-11T01:14:46.838948+0000","last_peered":"2024-05-11T01:14:46.838948+0000","last_clean":"2024-05-11T01:14:46.838948+0000","last_became_active":"2024-05-11T01:13:59.435148+0000","last_became_peered":"2024-05-11T01:13:59.435148+0000","last_unstale":"2024-05-11T01:14:46.838948+0000","last_undegraded":"2024-05-11T01:14:46.838948+0000","last_fullsized":"2024-05-11T01:14:46.838948+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:14:43.435242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837071+0000","last_change":"2024-05-11T01:13:59.438524+0000","last_active":"2024-05-11T01:14:46.837071+0000","last_peered":"2024-05-11T01:14:46.837071+0000","last_clean":"2024-05-11T01:14:46.837071+0000","last_became_active":"2024-05-11T01:13:59.438262+0000","last_became_peered":"2024-05-11T01:13:59.438262+0000","last_unstale":"2024-05-11T01:14:46.837071+0000","last_undegraded":"2024-05-11T01:14:46.837071+0000","last_fullsized":"2024-05-11T01:14:46.837071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:44:46.930438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854377+0000","last_change":"2024-05-11T01:13:59.432090+0000","last_active":"2024-05-11T01:14:46.854377+0000","last_peered":"2024-05-11T01:14:46.854377+0000","last_clean":"2024-05-11T01:14:46.854377+0000","last_became_active":"2024-05-11T01:13:59.431691+0000","last_became_peered":"2024-05-11T01:13:59.431691+0000","last_unstale":"2024-05-11T01:14:46.854377+0000","last_undegraded":"2024-05-11T01:14:46.854377+0000","last_fullsized":"2024-05-11T01:14:46.854377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:11:10.857350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836734+0000","last_change":"2024-05-11T01:13:59.440442+0000","last_active":"2024-05-11T01:14:46.836734+0000","last_peered":"2024-05-11T01:14:46.836734+0000","last_clean":"2024-05-11T01:14:46.836734+0000","last_became_active":"2024-05-11T01:13:59.440107+0000","last_became_peered":"2024-05-11T01:13:59.440107+0000","last_unstale":"2024-05-11T01:14:46.836734+0000","last_undegraded":"2024-05-11T01:14:46.836734+0000","last_fullsized":"2024-05-11T01:14:46.836734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:58:40.948977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837590+0000","last_change":"2024-05-11T01:13:58.440158+0000","last_active":"2024-05-11T01:14:46.837590+0000","last_peered":"2024-05-11T01:14:46.837590+0000","last_clean":"2024-05-11T01:14:46.837590+0000","last_became_active":"2024-05-11T01:13:58.439841+0000","last_became_peered":"2024-05-11T01:13:58.439841+0000","last_unstale":"2024-05-11T01:14:46.837590+0000","last_undegraded":"2024-05-11T01:14:46.837590+0000","last_fullsized":"2024-05-11T01:14:46.837590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837594+0000","last_change":"2024-05-11T01:13:59.433723+0000","last_active":"2024-05-11T01:14:46.837594+0000","last_peered":"2024-05-11T01:14:46.837594+0000","last_clean":"2024-05-11T01:14:46.837594+0000","last_became_active":"2024-05-11T01:13:59.433441+0000","last_became_peered":"2024-05-11T01:13:59.433441+0000","last_unstale":"2024-05-11T01:14:46.837594+0000","last_undegraded":"2024-05-11T01:14:46.837594+0000","last_fullsized":"2024-05-11T01:14:46.837594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839249+0000","last_change":"2024-05-11T01:13:58.433974+0000","last_active":"2024-05-11T01:14:46.839249+0000","last_peered":"2024-05-11T01:14:46.839249+0000","last_clean":"2024-05-11T01:14:46.839249+0000","last_became_active":"2024-05-11T01:13:58.433649+0000","last_became_peered":"2024-05-11T01:13:58.433649+0000","last_unstale":"2024-05-11T01:14:46.839249+0000","last_undegraded":"2024-05-11T01:14:46.839249+0000","last_fullsized":"2024-05-11T01:14:46.839249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:59.322532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853811+0000","last_change":"2024-05-11T01:13:59.439364+0000","last_active":"2024-05-11T01:14:46.853811+0000","last_peered":"2024-05-11T01:14:46.853811+0000","last_clean":"2024-05-11T01:14:46.853811+0000","last_became_active":"2024-05-11T01:13:59.439073+0000","last_became_peered":"2024-05-11T01:13:59.439073+0000","last_unstale":"2024-05-11T01:14:46.853811+0000","last_undegraded":"2024-05-11T01:14:46.853811+0000","last_fullsized":"2024-05-11T01:14:46.853811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:48.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837019+0000","last_change":"2024-05-11T01:13:58.435449+0000","last_active":"2024-05-11T01:14:46.837019+0000","last_peered":"2024-05-11T01:14:46.837019+0000","last_clean":"2024-05-11T01:14:46.837019+0000","last_became_active":"2024-05-11T01:13:58.434345+0000","last_became_peered":"2024-05-11T01:13:58.434345+0000","last_unstale":"2024-05-11T01:14:46.837019+0000","last_undegraded":"2024-05-11T01:14:46.837019+0000","last_fullsized":"2024-05-11T01:14:46.837019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:08:58.718268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836974+0000","last_change":"2024-05-11T01:13:59.440315+0000","last_active":"2024-05-11T01:14:46.836974+0000","last_peered":"2024-05-11T01:14:46.836974+0000","last_clean":"2024-05-11T01:14:46.836974+0000","last_became_active":"2024-05-11T01:13:59.439153+0000","last_became_peered":"2024-05-11T01:13:59.439153+0000","last_unstale":"2024-05-11T01:14:46.836974+0000","last_undegraded":"2024-05-11T01:14:46.836974+0000","last_fullsized":"2024-05-11T01:14:46.836974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:38:04.924508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854374+0000","last_change":"2024-05-11T01:13:58.434357+0000","last_active":"2024-05-11T01:14:46.854374+0000","last_peered":"2024-05-11T01:14:46.854374+0000","last_clean":"2024-05-11T01:14:46.854374+0000","last_became_active":"2024-05-11T01:13:58.433392+0000","last_became_peered":"2024-05-11T01:13:58.433392+0000","last_unstale":"2024-05-11T01:14:46.854374+0000","last_undegraded":"2024-05-11T01:14:46.854374+0000","last_fullsized":"2024-05-11T01:14:46.854374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:14.276107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838856+0000","last_change":"2024-05-11T01:13:59.435145+0000","last_active":"2024-05-11T01:14:46.838856+0000","last_peered":"2024-05-11T01:14:46.838856+0000","last_clean":"2024-05-11T01:14:46.838856+0000","last_became_active":"2024-05-11T01:13:59.434862+0000","last_became_peered":"2024-05-11T01:13:59.434862+0000","last_unstale":"2024-05-11T01:14:46.838856+0000","last_undegraded":"2024-05-11T01:14:46.838856+0000","last_fullsized":"2024-05-11T01:14:46.838856+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:17.395853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837423+0000","last_change":"2024-05-11T01:13:58.439588+0000","last_active":"2024-05-11T01:14:46.837423+0000","last_peered":"2024-05-11T01:14:46.837423+0000","last_clean":"2024-05-11T01:14:46.837423+0000","last_became_active":"2024-05-11T01:13:58.439133+0000","last_became_peered":"2024-05-11T01:13:58.439133+0000","last_unstale":"2024-05-11T01:14:46.837423+0000","last_undegraded":"2024-05-11T01:14:46.837423+0000","last_fullsized":"2024-05-11T01:14:46.837423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854573+0000","last_change":"2024-05-11T01:13:59.448882+0000","last_active":"2024-05-11T01:14:46.854573+0000","last_peered":"2024-05-11T01:14:46.854573+0000","last_clean":"2024-05-11T01:14:46.854573+0000","last_became_active":"2024-05-11T01:13:59.448208+0000","last_became_peered":"2024-05-11T01:13:59.448208+0000","last_unstale":"2024-05-11T01:14:46.854573+0000","last_undegraded":"2024-05-11T01:14:46.854573+0000","last_fullsized":"2024-05-11T01:14:46.854573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839012+0000","last_change":"2024-05-11T01:13:58.428838+0000","last_active":"2024-05-11T01:14:46.839012+0000","last_peered":"2024-05-11T01:14:46.839012+0000","last_clean":"2024-05-11T01:14:46.839012+0000","last_became_active":"2024-05-11T01:13:58.428556+0000","last_became_peered":"2024-05-11T01:13:58.428556+0000","last_unstale":"2024-05-11T01:14:46.839012+0000","last_undegraded":"2024-05-11T01:14:46.839012+0000","last_fullsized":"2024-05-11T01:14:46.839012+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:01:35.971417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854781+0000","last_change":"2024-05-11T01:13:59.447010+0000","last_active":"2024-05-11T01:14:46.854781+0000","last_peered":"2024-05-11T01:14:46.854781+0000","last_clean":"2024-05-11T01:14:46.854781+0000","last_became_active":"2024-05-11T01:13:59.446713+0000","last_became_peered":"2024-05-11T01:13:59.446713+0000","last_unstale":"2024-05-11T01:14:46.854781+0000","last_undegraded":"2024-05-11T01:14:46.854781+0000","last_fullsized":"2024-05-11T01:14:46.854781+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:03:04.844818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853638+0000","last_change":"2024-05-11T01:13:58.428031+0000","last_active":"2024-05-11T01:14:46.853638+0000","last_peered":"2024-05-11T01:14:46.853638+0000","last_clean":"2024-05-11T01:14:46.853638+0000","last_became_active":"2024-05-11T01:13:58.427742+0000","last_became_peered":"2024-05-11T01:13:58.427742+0000","last_unstale":"2024-05-11T01:14:46.853638+0000","last_undegraded":"2024-05-11T01:14:46.853638+0000","last_fullsized":"2024-05-11T01:14:46.853638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836923+0000","last_change":"2024-05-11T01:13:59.431589+0000","last_active":"2024-05-11T01:14:46.836923+0000","last_peered":"2024-05-11T01:14:46.836923+0000","last_clean":"2024-05-11T01:14:46.836923+0000","last_became_active":"2024-05-11T01:13:59.431292+0000","last_became_peered":"2024-05-11T01:13:59.431292+0000","last_unstale":"2024-05-11T01:14:46.836923+0000","last_undegraded":"2024-05-11T01:14:46.836923+0000","last_fullsized":"2024-05-11T01:14:46.836923+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853944+0000","last_change":"2024-05-11T01:13:58.436246+0000","last_active":"2024-05-11T01:14:46.853944+0000","last_peered":"2024-05-11T01:14:46.853944+0000","last_clean":"2024-05-11T01:14:46.853944+0000","last_became_active":"2024-05-11T01:13:58.435967+0000","last_became_peered":"2024-05-11T01:13:58.435967+0000","last_unstale":"2024-05-11T01:14:46.853944+0000","last_undegraded":"2024-05-11T01:14:46.853944+0000","last_fullsized":"2024-05-11T01:14:46.853944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:22.242555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839175+0000","last_change":"2024-05-11T01:13:59.445506+0000","last_active":"2024-05-11T01:14:46.839175+0000","last_peered":"2024-05-11T01:14:46.839175+0000","last_clean":"2024-05-11T01:14:46.839175+0000","last_became_active":"2024-05-11T01:13:59.445232+0000","last_became_peered":"2024-05-11T01:13:59.445232+0000","last_unstale":"2024-05-11T01:14:46.839175+0000","last_undegraded":"2024-05-11T01:14:46.839175+0000","last_fullsized":"2024-05-11T01:14:46.839175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:18:27.371597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839300+0000","last_change":"2024-05-11T01:13:58.426187+0000","last_active":"2024-05-11T01:14:46.839300+0000","last_peered":"2024-05-11T01:14:46.839300+0000","last_clean":"2024-05-11T01:14:46.839300+0000","last_became_active":"2024-05-11T01:13:58.425913+0000","last_became_peered":"2024-05-11T01:13:58.425913+0000","last_unstale":"2024-05-11T01:14:46.839300+0000","last_undegraded":"2024-05-11T01:14:46.839300+0000","last_fullsized":"2024-05-11T01:14:46.839300+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854582+0000","last_change":"2024-05-11T01:13:59.431401+0000","last_active":"2024-05-11T01:14:46.854582+0000","last_peered":"2024-05-11T01:14:46.854582+0000","last_clean":"2024-05-11T01:14:46.854582+0000","last_became_active":"2024-05-11T01:13:59.431126+0000","last_became_peered":"2024-05-11T01:13:59.431126+0000","last_unstale":"2024-05-11T01:14:46.854582+0000","last_undegraded":"2024-05-11T01:14:46.854582+0000","last_fullsized":"2024-05-11T01:14:46.854582+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:42:47.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855202+0000","last_change":"2024-05-11T01:13:58.431417+0000","last_active":"2024-05-11T01:14:46.855202+0000","last_peered":"2024-05-11T01:14:46.855202+0000","last_clean":"2024-05-11T01:14:46.855202+0000","last_became_active":"2024-05-11T01:13:58.431132+0000","last_became_peered":"2024-05-11T01:13:58.431132+0000","last_unstale":"2024-05-11T01:14:46.855202+0000","last_undegraded":"2024-05-11T01:14:46.855202+0000","last_fullsized":"2024-05-11T01:14:46.855202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:23:16.977274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836973+0000","last_change":"2024-05-11T01:13:59.436396+0000","last_active":"2024-05-11T01:14:46.836973+0000","last_peered":"2024-05-11T01:14:46.836973+0000","last_clean":"2024-05-11T01:14:46.836973+0000","last_became_active":"2024-05-11T01:13:59.436043+0000","last_became_peered":"2024-05-11T01:13:59.436043+0000","last_unstale":"2024-05-11T01:14:46.836973+0000","last_undegraded":"2024-05-11T01:14:46.836973+0000","last_fullsized":"2024-05-11T01:14:46.836973+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854476+0000","last_change":"2024-05-11T01:13:58.435566+0000","last_active":"2024-05-11T01:14:46.854476+0000","last_peered":"2024-05-11T01:14:46.854476+0000","last_clean":"2024-05-11T01:14:46.854476+0000","last_became_active":"2024-05-11T01:13:58.435280+0000","last_became_peered":"2024-05-11T01:13:58.435280+0000","last_unstale":"2024-05-11T01:14:46.854476+0000","last_undegraded":"2024-05-11T01:14:46.854476+0000","last_fullsized":"2024-05-11T01:14:46.854476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:47.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854368+0000","last_change":"2024-05-11T01:13:59.432307+0000","last_active":"2024-05-11T01:14:46.854368+0000","last_peered":"2024-05-11T01:14:46.854368+0000","last_clean":"2024-05-11T01:14:46.854368+0000","last_became_active":"2024-05-11T01:13:59.431913+0000","last_became_peered":"2024-05-11T01:13:59.431913+0000","last_unstale":"2024-05-11T01:14:46.854368+0000","last_undegraded":"2024-05-11T01:14:46.854368+0000","last_fullsized":"2024-05-11T01:14:46.854368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"63'10","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:07.973661+0000","last_change":"2024-05-11T01:13:58.425157+0000","last_active":"2024-05-11T01:16:07.973661+0000","last_peered":"2024-05-11T01:16:07.973661+0000","last_clean":"2024-05-11T01:16:07.973661+0000","last_became_active":"2024-05-11T01:13:58.424894+0000","last_became_peered":"2024-05-11T01:13:58.424894+0000","last_unstale":"2024-05-11T01:16:07.973661+0000","last_undegraded":"2024-05-11T01:16:07.973661+0000","last_fullsized":"2024-05-11T01:16:07.973661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:48:11.424880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837159+0000","last_change":"2024-05-11T01:13:59.447362+0000","last_active":"2024-05-11T01:14:46.837159+0000","last_peered":"2024-05-11T01:14:46.837159+0000","last_clean":"2024-05-11T01:14:46.837159+0000","last_became_active":"2024-05-11T01:13:59.447082+0000","last_became_peered":"2024-05-11T01:13:59.447082+0000","last_unstale":"2024-05-11T01:14:46.837159+0000","last_undegraded":"2024-05-11T01:14:46.837159+0000","last_fullsized":"2024-05-11T01:14:46.837159+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837499+0000","last_change":"2024-05-11T01:13:58.439044+0000","last_active":"2024-05-11T01:14:46.837499+0000","last_peered":"2024-05-11T01:14:46.837499+0000","last_clean":"2024-05-11T01:14:46.837499+0000","last_became_active":"2024-05-11T01:13:58.438603+0000","last_became_peered":"2024-05-11T01:13:58.438603+0000","last_unstale":"2024-05-11T01:14:46.837499+0000","last_undegraded":"2024-05-11T01:14:46.837499+0000","last_fullsized":"2024-05-11T01:14:46.837499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854651+0000","last_change":"2024-05-11T01:13:59.449114+0000","last_active":"2024-05-11T01:14:46.854651+0000","last_peered":"2024-05-11T01:14:46.854651+0000","last_clean":"2024-05-11T01:14:46.854651+0000","last_became_active":"2024-05-11T01:13:59.448532+0000","last_became_peered":"2024-05-11T01:13:59.448532+0000","last_unstale":"2024-05-11T01:14:46.854651+0000","last_undegraded":"2024-05-11T01:14:46.854651+0000","last_fullsized":"2024-05-11T01:14:46.854651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854151+0000","last_change":"2024-05-11T01:13:58.434118+0000","last_active":"2024-05-11T01:14:46.854151+0000","last_peered":"2024-05-11T01:14:46.854151+0000","last_clean":"2024-05-11T01:14:46.854151+0000","last_became_active":"2024-05-11T01:13:58.433818+0000","last_became_peered":"2024-05-11T01:13:58.433818+0000","last_unstale":"2024-05-11T01:14:46.854151+0000","last_undegraded":"2024-05-11T01:14:46.854151+0000","last_fullsized":"2024-05-11T01:14:46.854151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:49.867304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836702+0000","last_change":"2024-05-11T01:13:59.435013+0000","last_active":"2024-05-11T01:14:46.836702+0000","last_peered":"2024-05-11T01:14:46.836702+0000","last_clean":"2024-05-11T01:14:46.836702+0000","last_became_active":"2024-05-11T01:13:59.434726+0000","last_became_peered":"2024-05-11T01:13:59.434726+0000","last_unstale":"2024-05-11T01:14:46.836702+0000","last_undegraded":"2024-05-11T01:14:46.836702+0000","last_fullsized":"2024-05-11T01:14:46.836702+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:03:22.763694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837125+0000","last_change":"2024-05-11T01:13:58.438894+0000","last_active":"2024-05-11T01:14:46.837125+0000","last_peered":"2024-05-11T01:14:46.837125+0000","last_clean":"2024-05-11T01:14:46.837125+0000","last_became_active":"2024-05-11T01:13:58.438528+0000","last_became_peered":"2024-05-11T01:13:58.438528+0000","last_unstale":"2024-05-11T01:14:46.837125+0000","last_undegraded":"2024-05-11T01:14:46.837125+0000","last_fullsized":"2024-05-11T01:14:46.837125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854117+0000","last_change":"2024-05-11T01:13:59.437777+0000","last_active":"2024-05-11T01:14:46.854117+0000","last_peered":"2024-05-11T01:14:46.854117+0000","last_clean":"2024-05-11T01:14:46.854117+0000","last_became_active":"2024-05-11T01:13:59.437176+0000","last_became_peered":"2024-05-11T01:13:59.437176+0000","last_unstale":"2024-05-11T01:14:46.854117+0000","last_undegraded":"2024-05-11T01:14:46.854117+0000","last_fullsized":"2024-05-11T01:14:46.854117+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:07:21.319542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854062+0000","last_change":"2024-05-11T01:13:58.431218+0000","last_active":"2024-05-11T01:14:46.854062+0000","last_peered":"2024-05-11T01:14:46.854062+0000","last_clean":"2024-05-11T01:14:46.854062+0000","last_became_active":"2024-05-11T01:13:58.430952+0000","last_became_peered":"2024-05-11T01:13:58.430952+0000","last_unstale":"2024-05-11T01:14:46.854062+0000","last_undegraded":"2024-05-11T01:14:46.854062+0000","last_fullsized":"2024-05-11T01:14:46.854062+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:03:39.335329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854664+0000","last_change":"2024-05-11T01:13:59.427029+0000","last_active":"2024-05-11T01:14:46.854664+0000","last_peered":"2024-05-11T01:14:46.854664+0000","last_clean":"2024-05-11T01:14:46.854664+0000","last_became_active":"2024-05-11T01:13:59.426727+0000","last_became_peered":"2024-05-11T01:13:59.426727+0000","last_unstale":"2024-05-11T01:14:46.854664+0000","last_undegraded":"2024-05-11T01:14:46.854664+0000","last_fullsized":"2024-05-11T01:14:46.854664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:52.626350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854202+0000","last_change":"2024-05-11T01:13:58.434507+0000","last_active":"2024-05-11T01:14:46.854202+0000","last_peered":"2024-05-11T01:14:46.854202+0000","last_clean":"2024-05-11T01:14:46.854202+0000","last_became_active":"2024-05-11T01:13:58.434215+0000","last_became_peered":"2024-05-11T01:13:58.434215+0000","last_unstale":"2024-05-11T01:14:46.854202+0000","last_undegraded":"2024-05-11T01:14:46.854202+0000","last_fullsized":"2024-05-11T01:14:46.854202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:35.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854789+0000","last_change":"2024-05-11T01:13:59.444388+0000","last_active":"2024-05-11T01:14:46.854789+0000","last_peered":"2024-05-11T01:14:46.854789+0000","last_clean":"2024-05-11T01:14:46.854789+0000","last_became_active":"2024-05-11T01:13:59.443981+0000","last_became_peered":"2024-05-11T01:13:59.443981+0000","last_unstale":"2024-05-11T01:14:46.854789+0000","last_undegraded":"2024-05-11T01:14:46.854789+0000","last_fullsized":"2024-05-11T01:14:46.854789+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839290+0000","last_change":"2024-05-11T01:13:58.429801+0000","last_active":"2024-05-11T01:14:46.839290+0000","last_peered":"2024-05-11T01:14:46.839290+0000","last_clean":"2024-05-11T01:14:46.839290+0000","last_became_active":"2024-05-11T01:13:58.429538+0000","last_became_peered":"2024-05-11T01:13:58.429538+0000","last_unstale":"2024-05-11T01:14:46.839290+0000","last_undegraded":"2024-05-11T01:14:46.839290+0000","last_fullsized":"2024-05-11T01:14:46.839290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:27.314762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837321+0000","last_change":"2024-05-11T01:13:59.438763+0000","last_active":"2024-05-11T01:14:46.837321+0000","last_peered":"2024-05-11T01:14:46.837321+0000","last_clean":"2024-05-11T01:14:46.837321+0000","last_became_active":"2024-05-11T01:13:59.438474+0000","last_became_peered":"2024-05-11T01:13:59.438474+0000","last_unstale":"2024-05-11T01:14:46.837321+0000","last_undegraded":"2024-05-11T01:14:46.837321+0000","last_fullsized":"2024-05-11T01:14:46.837321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854295+0000","last_change":"2024-05-11T01:13:58.426925+0000","last_active":"2024-05-11T01:14:46.854295+0000","last_peered":"2024-05-11T01:14:46.854295+0000","last_clean":"2024-05-11T01:14:46.854295+0000","last_became_active":"2024-05-11T01:13:58.426586+0000","last_became_peered":"2024-05-11T01:13:58.426586+0000","last_unstale":"2024-05-11T01:14:46.854295+0000","last_undegraded":"2024-05-11T01:14:46.854295+0000","last_fullsized":"2024-05-11T01:14:46.854295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837463+0000","last_change":"2024-05-11T01:13:59.436970+0000","last_active":"2024-05-11T01:14:46.837463+0000","last_peered":"2024-05-11T01:14:46.837463+0000","last_clean":"2024-05-11T01:14:46.837463+0000","last_became_active":"2024-05-11T01:13:59.436373+0000","last_became_peered":"2024-05-11T01:13:59.436373+0000","last_unstale":"2024-05-11T01:14:46.837463+0000","last_undegraded":"2024-05-11T01:14:46.837463+0000","last_fullsized":"2024-05-11T01:14:46.837463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837079+0000","last_change":"2024-05-11T01:13:58.439680+0000","last_active":"2024-05-11T01:14:46.837079+0000","last_peered":"2024-05-11T01:14:46.837079+0000","last_clean":"2024-05-11T01:14:46.837079+0000","last_became_active":"2024-05-11T01:13:58.439404+0000","last_became_peered":"2024-05-11T01:13:58.439404+0000","last_unstale":"2024-05-11T01:14:46.837079+0000","last_undegraded":"2024-05-11T01:14:46.837079+0000","last_fullsized":"2024-05-11T01:14:46.837079+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:41:26.236385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837268+0000","last_change":"2024-05-11T01:13:59.435753+0000","last_active":"2024-05-11T01:14:46.837268+0000","last_peered":"2024-05-11T01:14:46.837268+0000","last_clean":"2024-05-11T01:14:46.837268+0000","last_became_active":"2024-05-11T01:13:59.435253+0000","last_became_peered":"2024-05-11T01:13:59.435253+0000","last_unstale":"2024-05-11T01:14:46.837268+0000","last_undegraded":"2024-05-11T01:14:46.837268+0000","last_fullsized":"2024-05-11T01:14:46.837268+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:06:58.758258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837593+0000","last_change":"2024-05-11T01:13:58.437862+0000","last_active":"2024-05-11T01:14:46.837593+0000","last_peered":"2024-05-11T01:14:46.837593+0000","last_clean":"2024-05-11T01:14:46.837593+0000","last_became_active":"2024-05-11T01:13:58.437597+0000","last_became_peered":"2024-05-11T01:13:58.437597+0000","last_unstale":"2024-05-11T01:14:46.837593+0000","last_undegraded":"2024-05-11T01:14:46.837593+0000","last_fullsized":"2024-05-11T01:14:46.837593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:19:23.818943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836437+0000","last_change":"2024-05-11T01:13:59.431673+0000","last_active":"2024-05-11T01:14:46.836437+0000","last_peered":"2024-05-11T01:14:46.836437+0000","last_clean":"2024-05-11T01:14:46.836437+0000","last_became_active":"2024-05-11T01:13:59.431386+0000","last_became_peered":"2024-05-11T01:13:59.431386+0000","last_unstale":"2024-05-11T01:14:46.836437+0000","last_undegraded":"2024-05-11T01:14:46.836437+0000","last_fullsized":"2024-05-11T01:14:46.836437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:27.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853615+0000","last_change":"2024-05-11T01:13:58.431189+0000","last_active":"2024-05-11T01:14:46.853615+0000","last_peered":"2024-05-11T01:14:46.853615+0000","last_clean":"2024-05-11T01:14:46.853615+0000","last_became_active":"2024-05-11T01:13:58.430890+0000","last_became_peered":"2024-05-11T01:13:58.430890+0000","last_unstale":"2024-05-11T01:14:46.853615+0000","last_undegraded":"2024-05-11T01:14:46.853615+0000","last_fullsized":"2024-05-11T01:14:46.853615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854490+0000","last_change":"2024-05-11T01:13:59.429997+0000","last_active":"2024-05-11T01:14:46.854490+0000","last_peered":"2024-05-11T01:14:46.854490+0000","last_clean":"2024-05-11T01:14:46.854490+0000","last_became_active":"2024-05-11T01:13:59.429734+0000","last_became_peered":"2024-05-11T01:13:59.429734+0000","last_unstale":"2024-05-11T01:14:46.854490+0000","last_undegraded":"2024-05-11T01:14:46.854490+0000","last_fullsized":"2024-05-11T01:14:46.854490+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:47.445191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854269+0000","last_change":"2024-05-11T01:13:58.431136+0000","last_active":"2024-05-11T01:14:46.854269+0000","last_peered":"2024-05-11T01:14:46.854269+0000","last_clean":"2024-05-11T01:14:46.854269+0000","last_became_active":"2024-05-11T01:13:58.430846+0000","last_became_peered":"2024-05-11T01:13:58.430846+0000","last_unstale":"2024-05-11T01:14:46.854269+0000","last_undegraded":"2024-05-11T01:14:46.854269+0000","last_fullsized":"2024-05-11T01:14:46.854269+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:38.439492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854167+0000","last_change":"2024-05-11T01:13:59.450616+0000","last_active":"2024-05-11T01:14:46.854167+0000","last_peered":"2024-05-11T01:14:46.854167+0000","last_clean":"2024-05-11T01:14:46.854167+0000","last_became_active":"2024-05-11T01:13:59.450300+0000","last_became_peered":"2024-05-11T01:13:59.450300+0000","last_unstale":"2024-05-11T01:14:46.854167+0000","last_undegraded":"2024-05-11T01:14:46.854167+0000","last_fullsized":"2024-05-11T01:14:46.854167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854250+0000","last_change":"2024-05-11T01:13:58.436187+0000","last_active":"2024-05-11T01:14:46.854250+0000","last_peered":"2024-05-11T01:14:46.854250+0000","last_clean":"2024-05-11T01:14:46.854250+0000","last_became_active":"2024-05-11T01:13:58.434702+0000","last_became_peered":"2024-05-11T01:13:58.434702+0000","last_unstale":"2024-05-11T01:14:46.854250+0000","last_undegraded":"2024-05-11T01:14:46.854250+0000","last_fullsized":"2024-05-11T01:14:46.854250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:28.737362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837639+0000","last_change":"2024-05-11T01:13:59.452028+0000","last_active":"2024-05-11T01:14:46.837639+0000","last_peered":"2024-05-11T01:14:46.837639+0000","last_clean":"2024-05-11T01:14:46.837639+0000","last_became_active":"2024-05-11T01:13:59.451773+0000","last_became_peered":"2024-05-11T01:13:59.451773+0000","last_unstale":"2024-05-11T01:14:46.837639+0000","last_undegraded":"2024-05-11T01:14:46.837639+0000","last_fullsized":"2024-05-11T01:14:46.837639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"61'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837109+0000","last_change":"2024-05-11T01:13:58.436921+0000","last_active":"2024-05-11T01:14:46.837109+0000","last_peered":"2024-05-11T01:14:46.837109+0000","last_clean":"2024-05-11T01:14:46.837109+0000","last_became_active":"2024-05-11T01:13:58.436650+0000","last_became_peered":"2024-05-11T01:13:58.436650+0000","last_unstale":"2024-05-11T01:14:46.837109+0000","last_undegraded":"2024-05-11T01:14:46.837109+0000","last_fullsized":"2024-05-11T01:14:46.837109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:16:06.268571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854569+0000","last_change":"2024-05-11T01:13:59.442952+0000","last_active":"2024-05-11T01:14:46.854569+0000","last_peered":"2024-05-11T01:14:46.854569+0000","last_clean":"2024-05-11T01:14:46.854569+0000","last_became_active":"2024-05-11T01:13:59.442413+0000","last_became_peered":"2024-05-11T01:13:59.442413+0000","last_unstale":"2024-05-11T01:14:46.854569+0000","last_undegraded":"2024-05-11T01:14:46.854569+0000","last_fullsized":"2024-05-11T01:14:46.854569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:32:54.808774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854522+0000","last_change":"2024-05-11T01:13:58.436342+0000","last_active":"2024-05-11T01:14:46.854522+0000","last_peered":"2024-05-11T01:14:46.854522+0000","last_clean":"2024-05-11T01:14:46.854522+0000","last_became_active":"2024-05-11T01:13:58.436037+0000","last_became_peered":"2024-05-11T01:13:58.436037+0000","last_unstale":"2024-05-11T01:14:46.854522+0000","last_undegraded":"2024-05-11T01:14:46.854522+0000","last_fullsized":"2024-05-11T01:14:46.854522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:22.498662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854431+0000","last_change":"2024-05-11T01:13:59.434239+0000","last_active":"2024-05-11T01:14:46.854431+0000","last_peered":"2024-05-11T01:14:46.854431+0000","last_clean":"2024-05-11T01:14:46.854431+0000","last_became_active":"2024-05-11T01:13:59.433946+0000","last_became_peered":"2024-05-11T01:13:59.433946+0000","last_unstale":"2024-05-11T01:14:46.854431+0000","last_undegraded":"2024-05-11T01:14:46.854431+0000","last_fullsized":"2024-05-11T01:14:46.854431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:07:36.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837338+0000","last_change":"2024-05-11T01:13:58.432458+0000","last_active":"2024-05-11T01:14:46.837338+0000","last_peered":"2024-05-11T01:14:46.837338+0000","last_clean":"2024-05-11T01:14:46.837338+0000","last_became_active":"2024-05-11T01:13:58.432149+0000","last_became_peered":"2024-05-11T01:13:58.432149+0000","last_unstale":"2024-05-11T01:14:46.837338+0000","last_undegraded":"2024-05-11T01:14:46.837338+0000","last_fullsized":"2024-05-11T01:14:46.837338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837389+0000","last_change":"2024-05-11T01:13:59.432345+0000","last_active":"2024-05-11T01:14:46.837389+0000","last_peered":"2024-05-11T01:14:46.837389+0000","last_clean":"2024-05-11T01:14:46.837389+0000","last_became_active":"2024-05-11T01:13:59.432058+0000","last_became_peered":"2024-05-11T01:13:59.432058+0000","last_unstale":"2024-05-11T01:14:46.837389+0000","last_undegraded":"2024-05-11T01:14:46.837389+0000","last_fullsized":"2024-05-11T01:14:46.837389+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:38:32.025100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854602+0000","last_change":"2024-05-11T01:13:58.433877+0000","last_active":"2024-05-11T01:14:46.854602+0000","last_peered":"2024-05-11T01:14:46.854602+0000","last_clean":"2024-05-11T01:14:46.854602+0000","last_became_active":"2024-05-11T01:13:58.433584+0000","last_became_peered":"2024-05-11T01:13:58.433584+0000","last_unstale":"2024-05-11T01:14:46.854602+0000","last_undegraded":"2024-05-11T01:14:46.854602+0000","last_fullsized":"2024-05-11T01:14:46.854602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:20.805314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837457+0000","last_change":"2024-05-11T01:13:59.434011+0000","last_active":"2024-05-11T01:14:46.837457+0000","last_peered":"2024-05-11T01:14:46.837457+0000","last_clean":"2024-05-11T01:14:46.837457+0000","last_became_active":"2024-05-11T01:13:59.432337+0000","last_became_peered":"2024-05-11T01:13:59.432337+0000","last_unstale":"2024-05-11T01:14:46.837457+0000","last_undegraded":"2024-05-11T01:14:46.837457+0000","last_fullsized":"2024-05-11T01:14:46.837457+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839100+0000","last_change":"2024-05-11T01:13:58.435211+0000","last_active":"2024-05-11T01:14:46.839100+0000","last_peered":"2024-05-11T01:14:46.839100+0000","last_clean":"2024-05-11T01:14:46.839100+0000","last_became_active":"2024-05-11T01:13:58.434956+0000","last_became_peered":"2024-05-11T01:13:58.434956+0000","last_unstale":"2024-05-11T01:14:46.839100+0000","last_undegraded":"2024-05-11T01:14:46.839100+0000","last_fullsized":"2024-05-11T01:14:46.839100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853560+0000","last_change":"2024-05-11T01:13:59.432319+0000","last_active":"2024-05-11T01:14:46.853560+0000","last_peered":"2024-05-11T01:14:46.853560+0000","last_clean":"2024-05-11T01:14:46.853560+0000","last_became_active":"2024-05-11T01:13:59.432012+0000","last_became_peered":"2024-05-11T01:13:59.432012+0000","last_unstale":"2024-05-11T01:14:46.853560+0000","last_undegraded":"2024-05-11T01:14:46.853560+0000","last_fullsized":"2024-05-11T01:14:46.853560+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:21.807568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836595+0000","last_change":"2024-05-11T01:13:58.433752+0000","last_active":"2024-05-11T01:14:46.836595+0000","last_peered":"2024-05-11T01:14:46.836595+0000","last_clean":"2024-05-11T01:14:46.836595+0000","last_became_active":"2024-05-11T01:13:58.433411+0000","last_became_peered":"2024-05-11T01:13:58.433411+0000","last_unstale":"2024-05-11T01:14:46.836595+0000","last_undegraded":"2024-05-11T01:14:46.836595+0000","last_fullsized":"2024-05-11T01:14:46.836595+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838906+0000","last_change":"2024-05-11T01:13:59.446971+0000","last_active":"2024-05-11T01:14:46.838906+0000","last_peered":"2024-05-11T01:14:46.838906+0000","last_clean":"2024-05-11T01:14:46.838906+0000","last_became_active":"2024-05-11T01:13:59.446670+0000","last_became_peered":"2024-05-11T01:13:59.446670+0000","last_unstale":"2024-05-11T01:14:46.838906+0000","last_undegraded":"2024-05-11T01:14:46.838906+0000","last_fullsized":"2024-05-11T01:14:46.838906+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:45:46.843918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854241+0000","last_change":"2024-05-11T01:13:58.435876+0000","last_active":"2024-05-11T01:14:46.854241+0000","last_peered":"2024-05-11T01:14:46.854241+0000","last_clean":"2024-05-11T01:14:46.854241+0000","last_became_active":"2024-05-11T01:13:58.434062+0000","last_became_peered":"2024-05-11T01:13:58.434062+0000","last_unstale":"2024-05-11T01:14:46.854241+0000","last_undegraded":"2024-05-11T01:14:46.854241+0000","last_fullsized":"2024-05-11T01:14:46.854241+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:12:26.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855069+0000","last_change":"2024-05-11T01:13:59.445426+0000","last_active":"2024-05-11T01:14:46.855069+0000","last_peered":"2024-05-11T01:14:46.855069+0000","last_clean":"2024-05-11T01:14:46.855069+0000","last_became_active":"2024-05-11T01:13:59.445146+0000","last_became_peered":"2024-05-11T01:13:59.445146+0000","last_unstale":"2024-05-11T01:14:46.855069+0000","last_undegraded":"2024-05-11T01:14:46.855069+0000","last_fullsized":"2024-05-11T01:14:46.855069+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:57:36.879340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854223+0000","last_change":"2024-05-11T01:13:58.434600+0000","last_active":"2024-05-11T01:14:46.854223+0000","last_peered":"2024-05-11T01:14:46.854223+0000","last_clean":"2024-05-11T01:14:46.854223+0000","last_became_active":"2024-05-11T01:13:58.434320+0000","last_became_peered":"2024-05-11T01:13:58.434320+0000","last_unstale":"2024-05-11T01:14:46.854223+0000","last_undegraded":"2024-05-11T01:14:46.854223+0000","last_fullsized":"2024-05-11T01:14:46.854223+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"51'86","reported_seq":246,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853980+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:14:46.853980+0000","last_peered":"2024-05-11T01:14:46.853980+0000","last_clean":"2024-05-11T01:14:46.853980+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:14:46.853980+0000","last_undegraded":"2024-05-11T01:14:46.853980+0000","last_fullsized":"2024-05-11T01:14:46.853980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3807,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":949409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542331,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":757760,"data_stored":359623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54128,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317712,"statfs":{"total":96636764160,"available":96581337088,"internally_reserved":0,"allocated":761856,"data_stored":361960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575033,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44408,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327432,"statfs":{"total":96636764160,"available":96591290368,"internally_reserved":0,"allocated":761856,"data_stored":359589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":765952,"data_stored":361982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58288,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313552,"statfs":{"total":96636764160,"available":96577077248,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575033,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1347584,"data_stored":949353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:16:15.541 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T01:16:15.541 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:16:15.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.680+0000 7f6fc255f640 1 Processor -- start 2024-05-11T01:16:15.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.680+0000 7f6fc255f640 1 -- start start 2024-05-11T01:16:15.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.680+0000 7f6fc255f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0d3330 0x7f6fbc0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.680+0000 7f6fc255f640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fbc0d1f90 con 0x7f6fbc15f2d0 2024-05-11T01:16:15.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.680+0000 7f6fc255f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fbc0d2270 con 0x7f6fbc0d2ea0 2024-05-11T01:16:15.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.680+0000 7f6fc255f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6fbc0d2550 con 0x7f6fbc0d3330 2024-05-11T01:16:15.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbbfff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f6fbc15f2d0 0x7f6fbc06e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33698/0 (socket says 172.21.15.52:33698) 2024-05-11T01:16:15.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbbfff640 1 -- 172.21.15.52:0/3826490748 learned_addr learned my addr 172.21.15.52:0/3826490748 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:15.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143759380 0 0) 0x7f6fbc0d1f90 con 0x7f6fbc15f2d0 2024-05-11T01:16:15.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fc0afc640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0d3330 0x7f6fbc0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fa4003520 con 0x7f6fbc15f2d0 2024-05-11T01:16:15.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 694797205 0 0) 0x7f6fa4003520 con 0x7f6fbc15f2d0 2024-05-11T01:16:15.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0d3330 msgr2=0x7f6fbc0b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:15.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0d3330 0x7f6fbc0b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 >> v1:172.21.15.119:6789/0 conn(0x7f6fbc0d2ea0 legacy=0x7f6fbc0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc0d2830 con 0x7f6fbc15f2d0 2024-05-11T01:16:15.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f6fac0016f0 con 0x7f6fbc15f2d0 2024-05-11T01:16:15.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 >> v1:172.21.15.52:6789/0 conn(0x7f6fbc15f2d0 legacy=0x7f6fbc06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fa4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fa4004800 0x7f6fa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0d3330 0x7f6fa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0d2550 con 0x7f6fa40039e0 2024-05-11T01:16:15.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa400ba70 con 0x7f6fa4004800 2024-05-11T01:16:15.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa400bc70 con 0x7f6fbc0d3330 2024-05-11T01:16:15.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbbfff640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fa4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.681+0000 7f6fbb7fe640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fa4004800 0x7f6fa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.682+0000 7f6fc0afc640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0d3330 0x7f6fa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.682+0000 7f6fbbfff640 1 -- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0d3330 msgr2=0x7f6fa40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:15.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.682+0000 7f6fbbfff640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0d3330 0x7f6fa40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.682+0000 7f6fbbfff640 1 -- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fa4004800 msgr2=0x7f6fa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:15.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.682+0000 7f6fbbfff640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fa4004800 0x7f6fa4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.682+0000 7f6fbbfff640 1 -- 172.21.15.52:0/3826490748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa4003520 con 0x7f6fa40039e0 2024-05-11T01:16:15.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.682+0000 7f6fbbfff640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fa4003dc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007fd0 tx=0x7f6fa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d18a216a43d6cc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:15.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac007ca0 con 0x7f6fa40039e0 2024-05-11T01:16:15.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac005ad0 con 0x7f6fa40039e0 2024-05-11T01:16:15.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fbaffd640 1 -- 172.21.15.52:0/3826490748 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac007070 con 0x7f6fa40039e0 2024-05-11T01:16:15.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fc255f640 1 -- 172.21.15.52:0/3826490748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 msgr2=0x7f6fa4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fc255f640 1 --2- 172.21.15.52:0/3826490748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fa4003dc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007fd0 tx=0x7f6fa4005830 comp rx=0 tx=0).stop 2024-05-11T01:16:15.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fc0afc640 1 -- 172.21.15.52:0/3826490748 reap_dead start 2024-05-11T01:16:15.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fc255f640 1 -- 172.21.15.52:0/3826490748 shutdown_connections 2024-05-11T01:16:15.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fc255f640 1 -- 172.21.15.52:0/3826490748 >> 172.21.15.52:0/3826490748 conn(0x7f6fbc09ba20 msgr2=0x7f6fbc09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:15.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fc255f640 1 -- 172.21.15.52:0/3826490748 shutdown_connections 2024-05-11T01:16:15.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.683+0000 7f6fc255f640 1 -- 172.21.15.52:0/3826490748 wait complete. 2024-05-11T01:16:15.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.684+0000 7f6fc255f640 1 Processor -- start 2024-05-11T01:16:15.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fc255f640 1 -- start start 2024-05-11T01:16:15.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fc255f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fbc14f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fc255f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fbc160cc0 0x7f6fbc161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fc255f640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0b2770 0x7f6fbc0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fc255f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc15f6b0 con 0x7f6fa40039e0 2024-05-11T01:16:15.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fc255f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0d3740 con 0x7f6fbc160cc0 2024-05-11T01:16:15.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fc255f640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f6fbc15f830 con 0x7f6fbc0b2770 2024-05-11T01:16:15.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fbbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fbc14f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fbbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fbc14f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43380/0 (socket says 172.21.15.52:43380) 2024-05-11T01:16:15.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.685+0000 7f6fbbfff640 1 -- 172.21.15.52:0/1017931139 learned_addr learned my addr 172.21.15.52:0/1017931139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:15.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fc0afc640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0b2770 0x7f6fbc0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fbb7fe640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fbc160cc0 0x7f6fbc161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fbbfff640 1 -- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0b2770 msgr2=0x7f6fbc0b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fbbfff640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0b2770 0x7f6fbc0b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fbbfff640 1 -- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fbc160cc0 msgr2=0x7f6fbc161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fbbfff640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fbc160cc0 0x7f6fbc161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fbbfff640 1 -- 172.21.15.52:0/1017931139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc155db0 con 0x7f6fa40039e0 2024-05-11T01:16:15.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fbb7fe640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fbc160cc0 0x7f6fbc161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:15.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fc0afc640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0b2770 0x7f6fbc0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:15.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fbbfff640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fbc14f5b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c250 tx=0x7f6fac001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:15.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fb8ff9640 1 -- 172.21.15.52:0/1017931139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0014b0 con 0x7f6fa40039e0 2024-05-11T01:16:15.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fb8ff9640 1 -- 172.21.15.52:0/1017931139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac038040 con 0x7f6fa40039e0 2024-05-11T01:16:15.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc14faf0 con 0x7f6fa40039e0 2024-05-11T01:16:15.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fb8ff9640 1 -- 172.21.15.52:0/1017931139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac007070 con 0x7f6fa40039e0 2024-05-11T01:16:15.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.686+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc1552d0 con 0x7f6fa40039e0 2024-05-11T01:16:15.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.687+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fbc0b3130 con 0x7f6fa40039e0 2024-05-11T01:16:15.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.688+0000 7f6fb8ff9640 1 -- 172.21.15.52:0/1017931139 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f6fac044030 con 0x7f6fa40039e0 2024-05-11T01:16:15.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.688+0000 7f6fb8ff9640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6fa8076300 0x7f6fa80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:15.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.688+0000 7f6fb8ff9640 1 -- 172.21.15.52:0/1017931139 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f6fac0b72a0 con 0x7f6fa40039e0 2024-05-11T01:16:15.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.689+0000 7f6fbb7fe640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6fa8076300 0x7f6fa80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:15.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.689+0000 7f6fbb7fe640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6fa8076300 0x7f6fa80787c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0c98d0 tx=0x7f6fb0009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:15.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.693+0000 7f6fb8ff9640 1 -- 172.21.15.52:0/1017931139 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6fac04b020 con 0x7f6fa40039e0 2024-05-11T01:16:15.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.884+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fbc069780 con 0x7f6fa8076300 2024-05-11T01:16:15.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.891+0000 7f6fb8ff9640 1 -- 172.21.15.52:0/1017931139 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251840 (secure 0 0 0) 0x7f6fbc069780 con 0x7f6fa8076300 2024-05-11T01:16:15.893 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:16:15.894 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:16:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6fa8076300 msgr2=0x7f6fa80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6fa8076300 0x7f6fa80787c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0c98d0 tx=0x7f6fb0009040 comp rx=0 tx=0).stop 2024-05-11T01:16:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 msgr2=0x7f6fbc14f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fbc14f5b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c250 tx=0x7f6fac001930 comp rx=0 tx=0).stop 2024-05-11T01:16:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 shutdown_connections 2024-05-11T01:16:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f6fa8076300 0x7f6fa80787c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6fbc0b2770 0x7f6fbc0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fbc160cc0 0x7f6fbc161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 --2- 172.21.15.52:0/1017931139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa40039e0 0x7f6fbc14f5b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 >> 172.21.15.52:0/1017931139 conn(0x7f6fbc09ba20 msgr2=0x7f6fbc0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 shutdown_connections 2024-05-11T01:16:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:15.894+0000 7f6fc255f640 1 -- 172.21.15.52:0/1017931139 wait complete. 2024-05-11T01:16:15.905 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-05-11T01:16:15.740321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594175,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":174,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317588,"kb_used_data":7080,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754657132,"statfs":{"total":773094113280,"available":772768903168,"internally_reserved":0,"allocated":7249920,"data_stored":4059958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":317836849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003244"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854522+0000","last_change":"2024-05-11T01:13:59.434893+0000","last_active":"2024-05-11T01:14:46.854522+0000","last_peered":"2024-05-11T01:14:46.854522+0000","last_clean":"2024-05-11T01:14:46.854522+0000","last_became_active":"2024-05-11T01:13:59.434594+0000","last_became_peered":"2024-05-11T01:13:59.434594+0000","last_unstale":"2024-05-11T01:14:46.854522+0000","last_undegraded":"2024-05-11T01:14:46.854522+0000","last_fullsized":"2024-05-11T01:14:46.854522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:49:33.685964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854143+0000","last_change":"2024-05-11T01:13:59.446282+0000","last_active":"2024-05-11T01:14:46.854143+0000","last_peered":"2024-05-11T01:14:46.854143+0000","last_clean":"2024-05-11T01:14:46.854143+0000","last_became_active":"2024-05-11T01:13:59.445971+0000","last_became_peered":"2024-05-11T01:13:59.445971+0000","last_unstale":"2024-05-11T01:14:46.854143+0000","last_undegraded":"2024-05-11T01:14:46.854143+0000","last_fullsized":"2024-05-11T01:14:46.854143+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:50.869559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839269+0000","last_change":"2024-05-11T01:13:59.430661+0000","last_active":"2024-05-11T01:14:46.839269+0000","last_peered":"2024-05-11T01:14:46.839269+0000","last_clean":"2024-05-11T01:14:46.839269+0000","last_became_active":"2024-05-11T01:13:59.430408+0000","last_became_peered":"2024-05-11T01:13:59.430408+0000","last_unstale":"2024-05-11T01:14:46.839269+0000","last_undegraded":"2024-05-11T01:14:46.839269+0000","last_fullsized":"2024-05-11T01:14:46.839269+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:14:03.379607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854537+0000","last_change":"2024-05-11T01:13:59.448424+0000","last_active":"2024-05-11T01:14:46.854537+0000","last_peered":"2024-05-11T01:14:46.854537+0000","last_clean":"2024-05-11T01:14:46.854537+0000","last_became_active":"2024-05-11T01:13:59.446986+0000","last_became_peered":"2024-05-11T01:13:59.446986+0000","last_unstale":"2024-05-11T01:14:46.854537+0000","last_undegraded":"2024-05-11T01:14:46.854537+0000","last_fullsized":"2024-05-11T01:14:46.854537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853610+0000","last_change":"2024-05-11T01:13:59.446369+0000","last_active":"2024-05-11T01:14:46.853610+0000","last_peered":"2024-05-11T01:14:46.853610+0000","last_clean":"2024-05-11T01:14:46.853610+0000","last_became_active":"2024-05-11T01:13:59.445922+0000","last_became_peered":"2024-05-11T01:13:59.445922+0000","last_unstale":"2024-05-11T01:14:46.853610+0000","last_undegraded":"2024-05-11T01:14:46.853610+0000","last_fullsized":"2024-05-11T01:14:46.853610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:15.278361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837202+0000","last_change":"2024-05-11T01:13:59.435535+0000","last_active":"2024-05-11T01:14:46.837202+0000","last_peered":"2024-05-11T01:14:46.837202+0000","last_clean":"2024-05-11T01:14:46.837202+0000","last_became_active":"2024-05-11T01:13:59.434634+0000","last_became_peered":"2024-05-11T01:13:59.434634+0000","last_unstale":"2024-05-11T01:14:46.837202+0000","last_undegraded":"2024-05-11T01:14:46.837202+0000","last_fullsized":"2024-05-11T01:14:46.837202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:34:16.451030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854001+0000","last_change":"2024-05-11T01:13:59.434552+0000","last_active":"2024-05-11T01:14:46.854001+0000","last_peered":"2024-05-11T01:14:46.854001+0000","last_clean":"2024-05-11T01:14:46.854001+0000","last_became_active":"2024-05-11T01:13:59.433976+0000","last_became_peered":"2024-05-11T01:13:59.433976+0000","last_unstale":"2024-05-11T01:14:46.854001+0000","last_undegraded":"2024-05-11T01:14:46.854001+0000","last_fullsized":"2024-05-11T01:14:46.854001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853803+0000","last_change":"2024-05-11T01:13:59.447658+0000","last_active":"2024-05-11T01:14:46.853803+0000","last_peered":"2024-05-11T01:14:46.853803+0000","last_clean":"2024-05-11T01:14:46.853803+0000","last_became_active":"2024-05-11T01:13:59.446507+0000","last_became_peered":"2024-05-11T01:13:59.446507+0000","last_unstale":"2024-05-11T01:14:46.853803+0000","last_undegraded":"2024-05-11T01:14:46.853803+0000","last_fullsized":"2024-05-11T01:14:46.853803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:34:51.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854683+0000","last_change":"2024-05-11T01:13:59.449973+0000","last_active":"2024-05-11T01:14:46.854683+0000","last_peered":"2024-05-11T01:14:46.854683+0000","last_clean":"2024-05-11T01:14:46.854683+0000","last_became_active":"2024-05-11T01:13:59.449670+0000","last_became_peered":"2024-05-11T01:13:59.449670+0000","last_unstale":"2024-05-11T01:14:46.854683+0000","last_undegraded":"2024-05-11T01:14:46.854683+0000","last_fullsized":"2024-05-11T01:14:46.854683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854986+0000","last_change":"2024-05-11T01:13:59.450134+0000","last_active":"2024-05-11T01:14:46.854986+0000","last_peered":"2024-05-11T01:14:46.854986+0000","last_clean":"2024-05-11T01:14:46.854986+0000","last_became_active":"2024-05-11T01:13:59.449822+0000","last_became_peered":"2024-05-11T01:13:59.449822+0000","last_unstale":"2024-05-11T01:14:46.854986+0000","last_undegraded":"2024-05-11T01:14:46.854986+0000","last_fullsized":"2024-05-11T01:14:46.854986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.840088+0000","last_change":"2024-05-11T01:14:46.840088+0000","last_active":"2024-05-11T01:14:46.840088+0000","last_peered":"2024-05-11T01:14:46.840088+0000","last_clean":"2024-05-11T01:14:46.840088+0000","last_became_active":"2024-05-11T01:14:46.839837+0000","last_became_peered":"2024-05-11T01:14:46.839837+0000","last_unstale":"2024-05-11T01:14:46.840088+0000","last_undegraded":"2024-05-11T01:14:46.840088+0000","last_fullsized":"2024-05-11T01:14:46.840088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:38:11.978713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853737+0000","last_change":"2024-05-11T01:13:59.428513+0000","last_active":"2024-05-11T01:14:46.853737+0000","last_peered":"2024-05-11T01:14:46.853737+0000","last_clean":"2024-05-11T01:14:46.853737+0000","last_became_active":"2024-05-11T01:13:59.428202+0000","last_became_peered":"2024-05-11T01:13:59.428202+0000","last_unstale":"2024-05-11T01:14:46.853737+0000","last_undegraded":"2024-05-11T01:14:46.853737+0000","last_fullsized":"2024-05-11T01:14:46.853737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:23.500916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854223+0000","last_change":"2024-05-11T01:13:59.435402+0000","last_active":"2024-05-11T01:14:46.854223+0000","last_peered":"2024-05-11T01:14:46.854223+0000","last_clean":"2024-05-11T01:14:46.854223+0000","last_became_active":"2024-05-11T01:13:59.433324+0000","last_became_peered":"2024-05-11T01:13:59.433324+0000","last_unstale":"2024-05-11T01:14:46.854223+0000","last_undegraded":"2024-05-11T01:14:46.854223+0000","last_fullsized":"2024-05-11T01:14:46.854223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:36:46.612605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854315+0000","last_change":"2024-05-11T01:13:59.447113+0000","last_active":"2024-05-11T01:14:46.854315+0000","last_peered":"2024-05-11T01:14:46.854315+0000","last_clean":"2024-05-11T01:14:46.854315+0000","last_became_active":"2024-05-11T01:13:59.446349+0000","last_became_peered":"2024-05-11T01:13:59.446349+0000","last_unstale":"2024-05-11T01:14:46.854315+0000","last_undegraded":"2024-05-11T01:14:46.854315+0000","last_fullsized":"2024-05-11T01:14:46.854315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836869+0000","last_change":"2024-05-11T01:13:59.437811+0000","last_active":"2024-05-11T01:14:46.836869+0000","last_peered":"2024-05-11T01:14:46.836869+0000","last_clean":"2024-05-11T01:14:46.836869+0000","last_became_active":"2024-05-11T01:13:59.427975+0000","last_became_peered":"2024-05-11T01:13:59.427975+0000","last_unstale":"2024-05-11T01:14:46.836869+0000","last_undegraded":"2024-05-11T01:14:46.836869+0000","last_fullsized":"2024-05-11T01:14:46.836869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:05:27.108182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836920+0000","last_change":"2024-05-11T01:13:59.446666+0000","last_active":"2024-05-11T01:14:46.836920+0000","last_peered":"2024-05-11T01:14:46.836920+0000","last_clean":"2024-05-11T01:14:46.836920+0000","last_became_active":"2024-05-11T01:13:59.446396+0000","last_became_peered":"2024-05-11T01:13:59.446396+0000","last_unstale":"2024-05-11T01:14:46.836920+0000","last_undegraded":"2024-05-11T01:14:46.836920+0000","last_fullsized":"2024-05-11T01:14:46.836920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:55:23.003515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839156+0000","last_change":"2024-05-11T01:13:59.437261+0000","last_active":"2024-05-11T01:14:46.839156+0000","last_peered":"2024-05-11T01:14:46.839156+0000","last_clean":"2024-05-11T01:14:46.839156+0000","last_became_active":"2024-05-11T01:13:59.436992+0000","last_became_peered":"2024-05-11T01:13:59.436992+0000","last_unstale":"2024-05-11T01:14:46.839156+0000","last_undegraded":"2024-05-11T01:14:46.839156+0000","last_fullsized":"2024-05-11T01:14:46.839156+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:53.116569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839026+0000","last_change":"2024-05-11T01:13:59.445508+0000","last_active":"2024-05-11T01:14:46.839026+0000","last_peered":"2024-05-11T01:14:46.839026+0000","last_clean":"2024-05-11T01:14:46.839026+0000","last_became_active":"2024-05-11T01:13:59.445318+0000","last_became_peered":"2024-05-11T01:13:59.445318+0000","last_unstale":"2024-05-11T01:14:46.839026+0000","last_undegraded":"2024-05-11T01:14:46.839026+0000","last_fullsized":"2024-05-11T01:14:46.839026+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:21.154101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855679+0000","last_change":"2024-05-11T01:14:46.855679+0000","last_active":"2024-05-11T01:14:46.855679+0000","last_peered":"2024-05-11T01:14:46.855679+0000","last_clean":"2024-05-11T01:14:46.855679+0000","last_became_active":"2024-05-11T01:14:46.855320+0000","last_became_peered":"2024-05-11T01:14:46.855320+0000","last_unstale":"2024-05-11T01:14:46.855679+0000","last_undegraded":"2024-05-11T01:14:46.855679+0000","last_fullsized":"2024-05-11T01:14:46.855679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:48:36.304487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854538+0000","last_change":"2024-05-11T01:13:59.444231+0000","last_active":"2024-05-11T01:14:46.854538+0000","last_peered":"2024-05-11T01:14:46.854538+0000","last_clean":"2024-05-11T01:14:46.854538+0000","last_became_active":"2024-05-11T01:13:59.443762+0000","last_became_peered":"2024-05-11T01:13:59.443762+0000","last_unstale":"2024-05-11T01:14:46.854538+0000","last_undegraded":"2024-05-11T01:14:46.854538+0000","last_fullsized":"2024-05-11T01:14:46.854538+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:25:17.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837269+0000","last_change":"2024-05-11T01:13:59.437781+0000","last_active":"2024-05-11T01:14:46.837269+0000","last_peered":"2024-05-11T01:14:46.837269+0000","last_clean":"2024-05-11T01:14:46.837269+0000","last_became_active":"2024-05-11T01:13:59.437444+0000","last_became_peered":"2024-05-11T01:13:59.437444+0000","last_unstale":"2024-05-11T01:14:46.837269+0000","last_undegraded":"2024-05-11T01:14:46.837269+0000","last_fullsized":"2024-05-11T01:14:46.837269+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:31:12.326102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854442+0000","last_change":"2024-05-11T01:13:59.447343+0000","last_active":"2024-05-11T01:14:46.854442+0000","last_peered":"2024-05-11T01:14:46.854442+0000","last_clean":"2024-05-11T01:14:46.854442+0000","last_became_active":"2024-05-11T01:13:59.446779+0000","last_became_peered":"2024-05-11T01:13:59.446779+0000","last_unstale":"2024-05-11T01:14:46.854442+0000","last_undegraded":"2024-05-11T01:14:46.854442+0000","last_fullsized":"2024-05-11T01:14:46.854442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836924+0000","last_change":"2024-05-11T01:13:59.431582+0000","last_active":"2024-05-11T01:14:46.836924+0000","last_peered":"2024-05-11T01:14:46.836924+0000","last_clean":"2024-05-11T01:14:46.836924+0000","last_became_active":"2024-05-11T01:13:59.431309+0000","last_became_peered":"2024-05-11T01:13:59.431309+0000","last_unstale":"2024-05-11T01:14:46.836924+0000","last_undegraded":"2024-05-11T01:14:46.836924+0000","last_fullsized":"2024-05-11T01:14:46.836924+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837032+0000","last_change":"2024-05-11T01:13:59.433902+0000","last_active":"2024-05-11T01:14:46.837032+0000","last_peered":"2024-05-11T01:14:46.837032+0000","last_clean":"2024-05-11T01:14:46.837032+0000","last_became_active":"2024-05-11T01:13:59.433606+0000","last_became_peered":"2024-05-11T01:13:59.433606+0000","last_unstale":"2024-05-11T01:14:46.837032+0000","last_undegraded":"2024-05-11T01:14:46.837032+0000","last_fullsized":"2024-05-11T01:14:46.837032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837537+0000","last_change":"2024-05-11T01:13:59.436026+0000","last_active":"2024-05-11T01:14:46.837537+0000","last_peered":"2024-05-11T01:14:46.837537+0000","last_clean":"2024-05-11T01:14:46.837537+0000","last_became_active":"2024-05-11T01:13:59.435708+0000","last_became_peered":"2024-05-11T01:13:59.435708+0000","last_unstale":"2024-05-11T01:14:46.837537+0000","last_undegraded":"2024-05-11T01:14:46.837537+0000","last_fullsized":"2024-05-11T01:14:46.837537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854605+0000","last_change":"2024-05-11T01:13:59.441132+0000","last_active":"2024-05-11T01:14:46.854605+0000","last_peered":"2024-05-11T01:14:46.854605+0000","last_clean":"2024-05-11T01:14:46.854605+0000","last_became_active":"2024-05-11T01:13:59.440852+0000","last_became_peered":"2024-05-11T01:13:59.440852+0000","last_unstale":"2024-05-11T01:14:46.854605+0000","last_undegraded":"2024-05-11T01:14:46.854605+0000","last_fullsized":"2024-05-11T01:14:46.854605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:18:58.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839326+0000","last_change":"2024-05-11T01:13:59.436547+0000","last_active":"2024-05-11T01:14:46.839326+0000","last_peered":"2024-05-11T01:14:46.839326+0000","last_clean":"2024-05-11T01:14:46.839326+0000","last_became_active":"2024-05-11T01:13:59.436260+0000","last_became_peered":"2024-05-11T01:13:59.436260+0000","last_unstale":"2024-05-11T01:14:46.839326+0000","last_undegraded":"2024-05-11T01:14:46.839326+0000","last_fullsized":"2024-05-11T01:14:46.839326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:27.176219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853867+0000","last_change":"2024-05-11T01:13:59.425177+0000","last_active":"2024-05-11T01:14:46.853867+0000","last_peered":"2024-05-11T01:14:46.853867+0000","last_clean":"2024-05-11T01:14:46.853867+0000","last_became_active":"2024-05-11T01:13:59.424328+0000","last_became_peered":"2024-05-11T01:13:59.424328+0000","last_unstale":"2024-05-11T01:14:46.853867+0000","last_undegraded":"2024-05-11T01:14:46.853867+0000","last_fullsized":"2024-05-11T01:14:46.853867+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:29.739616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837190+0000","last_change":"2024-05-11T01:13:59.436551+0000","last_active":"2024-05-11T01:14:46.837190+0000","last_peered":"2024-05-11T01:14:46.837190+0000","last_clean":"2024-05-11T01:14:46.837190+0000","last_became_active":"2024-05-11T01:13:59.436257+0000","last_became_peered":"2024-05-11T01:13:59.436257+0000","last_unstale":"2024-05-11T01:14:46.837190+0000","last_undegraded":"2024-05-11T01:14:46.837190+0000","last_fullsized":"2024-05-11T01:14:46.837190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:37:44.925797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838948+0000","last_change":"2024-05-11T01:13:59.435398+0000","last_active":"2024-05-11T01:14:46.838948+0000","last_peered":"2024-05-11T01:14:46.838948+0000","last_clean":"2024-05-11T01:14:46.838948+0000","last_became_active":"2024-05-11T01:13:59.435148+0000","last_became_peered":"2024-05-11T01:13:59.435148+0000","last_unstale":"2024-05-11T01:14:46.838948+0000","last_undegraded":"2024-05-11T01:14:46.838948+0000","last_fullsized":"2024-05-11T01:14:46.838948+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:14:43.435242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837071+0000","last_change":"2024-05-11T01:13:59.438524+0000","last_active":"2024-05-11T01:14:46.837071+0000","last_peered":"2024-05-11T01:14:46.837071+0000","last_clean":"2024-05-11T01:14:46.837071+0000","last_became_active":"2024-05-11T01:13:59.438262+0000","last_became_peered":"2024-05-11T01:13:59.438262+0000","last_unstale":"2024-05-11T01:14:46.837071+0000","last_undegraded":"2024-05-11T01:14:46.837071+0000","last_fullsized":"2024-05-11T01:14:46.837071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:44:46.930438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854377+0000","last_change":"2024-05-11T01:13:59.432090+0000","last_active":"2024-05-11T01:14:46.854377+0000","last_peered":"2024-05-11T01:14:46.854377+0000","last_clean":"2024-05-11T01:14:46.854377+0000","last_became_active":"2024-05-11T01:13:59.431691+0000","last_became_peered":"2024-05-11T01:13:59.431691+0000","last_unstale":"2024-05-11T01:14:46.854377+0000","last_undegraded":"2024-05-11T01:14:46.854377+0000","last_fullsized":"2024-05-11T01:14:46.854377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:11:10.857350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836734+0000","last_change":"2024-05-11T01:13:59.440442+0000","last_active":"2024-05-11T01:14:46.836734+0000","last_peered":"2024-05-11T01:14:46.836734+0000","last_clean":"2024-05-11T01:14:46.836734+0000","last_became_active":"2024-05-11T01:13:59.440107+0000","last_became_peered":"2024-05-11T01:13:59.440107+0000","last_unstale":"2024-05-11T01:14:46.836734+0000","last_undegraded":"2024-05-11T01:14:46.836734+0000","last_fullsized":"2024-05-11T01:14:46.836734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:58:40.948977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837590+0000","last_change":"2024-05-11T01:13:58.440158+0000","last_active":"2024-05-11T01:14:46.837590+0000","last_peered":"2024-05-11T01:14:46.837590+0000","last_clean":"2024-05-11T01:14:46.837590+0000","last_became_active":"2024-05-11T01:13:58.439841+0000","last_became_peered":"2024-05-11T01:13:58.439841+0000","last_unstale":"2024-05-11T01:14:46.837590+0000","last_undegraded":"2024-05-11T01:14:46.837590+0000","last_fullsized":"2024-05-11T01:14:46.837590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837594+0000","last_change":"2024-05-11T01:13:59.433723+0000","last_active":"2024-05-11T01:14:46.837594+0000","last_peered":"2024-05-11T01:14:46.837594+0000","last_clean":"2024-05-11T01:14:46.837594+0000","last_became_active":"2024-05-11T01:13:59.433441+0000","last_became_peered":"2024-05-11T01:13:59.433441+0000","last_unstale":"2024-05-11T01:14:46.837594+0000","last_undegraded":"2024-05-11T01:14:46.837594+0000","last_fullsized":"2024-05-11T01:14:46.837594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839249+0000","last_change":"2024-05-11T01:13:58.433974+0000","last_active":"2024-05-11T01:14:46.839249+0000","last_peered":"2024-05-11T01:14:46.839249+0000","last_clean":"2024-05-11T01:14:46.839249+0000","last_became_active":"2024-05-11T01:13:58.433649+0000","last_became_peered":"2024-05-11T01:13:58.433649+0000","last_unstale":"2024-05-11T01:14:46.839249+0000","last_undegraded":"2024-05-11T01:14:46.839249+0000","last_fullsized":"2024-05-11T01:14:46.839249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:59.322532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853811+0000","last_change":"2024-05-11T01:13:59.439364+0000","last_active":"2024-05-11T01:14:46.853811+0000","last_peered":"2024-05-11T01:14:46.853811+0000","last_clean":"2024-05-11T01:14:46.853811+0000","last_became_active":"2024-05-11T01:13:59.439073+0000","last_became_peered":"2024-05-11T01:13:59.439073+0000","last_unstale":"2024-05-11T01:14:46.853811+0000","last_undegraded":"2024-05-11T01:14:46.853811+0000","last_fullsized":"2024-05-11T01:14:46.853811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:48.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837019+0000","last_change":"2024-05-11T01:13:58.435449+0000","last_active":"2024-05-11T01:14:46.837019+0000","last_peered":"2024-05-11T01:14:46.837019+0000","last_clean":"2024-05-11T01:14:46.837019+0000","last_became_active":"2024-05-11T01:13:58.434345+0000","last_became_peered":"2024-05-11T01:13:58.434345+0000","last_unstale":"2024-05-11T01:14:46.837019+0000","last_undegraded":"2024-05-11T01:14:46.837019+0000","last_fullsized":"2024-05-11T01:14:46.837019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:08:58.718268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836974+0000","last_change":"2024-05-11T01:13:59.440315+0000","last_active":"2024-05-11T01:14:46.836974+0000","last_peered":"2024-05-11T01:14:46.836974+0000","last_clean":"2024-05-11T01:14:46.836974+0000","last_became_active":"2024-05-11T01:13:59.439153+0000","last_became_peered":"2024-05-11T01:13:59.439153+0000","last_unstale":"2024-05-11T01:14:46.836974+0000","last_undegraded":"2024-05-11T01:14:46.836974+0000","last_fullsized":"2024-05-11T01:14:46.836974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:38:04.924508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854374+0000","last_change":"2024-05-11T01:13:58.434357+0000","last_active":"2024-05-11T01:14:46.854374+0000","last_peered":"2024-05-11T01:14:46.854374+0000","last_clean":"2024-05-11T01:14:46.854374+0000","last_became_active":"2024-05-11T01:13:58.433392+0000","last_became_peered":"2024-05-11T01:13:58.433392+0000","last_unstale":"2024-05-11T01:14:46.854374+0000","last_undegraded":"2024-05-11T01:14:46.854374+0000","last_fullsized":"2024-05-11T01:14:46.854374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:14.276107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838856+0000","last_change":"2024-05-11T01:13:59.435145+0000","last_active":"2024-05-11T01:14:46.838856+0000","last_peered":"2024-05-11T01:14:46.838856+0000","last_clean":"2024-05-11T01:14:46.838856+0000","last_became_active":"2024-05-11T01:13:59.434862+0000","last_became_peered":"2024-05-11T01:13:59.434862+0000","last_unstale":"2024-05-11T01:14:46.838856+0000","last_undegraded":"2024-05-11T01:14:46.838856+0000","last_fullsized":"2024-05-11T01:14:46.838856+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:17.395853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837423+0000","last_change":"2024-05-11T01:13:58.439588+0000","last_active":"2024-05-11T01:14:46.837423+0000","last_peered":"2024-05-11T01:14:46.837423+0000","last_clean":"2024-05-11T01:14:46.837423+0000","last_became_active":"2024-05-11T01:13:58.439133+0000","last_became_peered":"2024-05-11T01:13:58.439133+0000","last_unstale":"2024-05-11T01:14:46.837423+0000","last_undegraded":"2024-05-11T01:14:46.837423+0000","last_fullsized":"2024-05-11T01:14:46.837423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854573+0000","last_change":"2024-05-11T01:13:59.448882+0000","last_active":"2024-05-11T01:14:46.854573+0000","last_peered":"2024-05-11T01:14:46.854573+0000","last_clean":"2024-05-11T01:14:46.854573+0000","last_became_active":"2024-05-11T01:13:59.448208+0000","last_became_peered":"2024-05-11T01:13:59.448208+0000","last_unstale":"2024-05-11T01:14:46.854573+0000","last_undegraded":"2024-05-11T01:14:46.854573+0000","last_fullsized":"2024-05-11T01:14:46.854573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839012+0000","last_change":"2024-05-11T01:13:58.428838+0000","last_active":"2024-05-11T01:14:46.839012+0000","last_peered":"2024-05-11T01:14:46.839012+0000","last_clean":"2024-05-11T01:14:46.839012+0000","last_became_active":"2024-05-11T01:13:58.428556+0000","last_became_peered":"2024-05-11T01:13:58.428556+0000","last_unstale":"2024-05-11T01:14:46.839012+0000","last_undegraded":"2024-05-11T01:14:46.839012+0000","last_fullsized":"2024-05-11T01:14:46.839012+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:01:35.971417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854781+0000","last_change":"2024-05-11T01:13:59.447010+0000","last_active":"2024-05-11T01:14:46.854781+0000","last_peered":"2024-05-11T01:14:46.854781+0000","last_clean":"2024-05-11T01:14:46.854781+0000","last_became_active":"2024-05-11T01:13:59.446713+0000","last_became_peered":"2024-05-11T01:13:59.446713+0000","last_unstale":"2024-05-11T01:14:46.854781+0000","last_undegraded":"2024-05-11T01:14:46.854781+0000","last_fullsized":"2024-05-11T01:14:46.854781+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:03:04.844818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853638+0000","last_change":"2024-05-11T01:13:58.428031+0000","last_active":"2024-05-11T01:14:46.853638+0000","last_peered":"2024-05-11T01:14:46.853638+0000","last_clean":"2024-05-11T01:14:46.853638+0000","last_became_active":"2024-05-11T01:13:58.427742+0000","last_became_peered":"2024-05-11T01:13:58.427742+0000","last_unstale":"2024-05-11T01:14:46.853638+0000","last_undegraded":"2024-05-11T01:14:46.853638+0000","last_fullsized":"2024-05-11T01:14:46.853638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836923+0000","last_change":"2024-05-11T01:13:59.431589+0000","last_active":"2024-05-11T01:14:46.836923+0000","last_peered":"2024-05-11T01:14:46.836923+0000","last_clean":"2024-05-11T01:14:46.836923+0000","last_became_active":"2024-05-11T01:13:59.431292+0000","last_became_peered":"2024-05-11T01:13:59.431292+0000","last_unstale":"2024-05-11T01:14:46.836923+0000","last_undegraded":"2024-05-11T01:14:46.836923+0000","last_fullsized":"2024-05-11T01:14:46.836923+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853944+0000","last_change":"2024-05-11T01:13:58.436246+0000","last_active":"2024-05-11T01:14:46.853944+0000","last_peered":"2024-05-11T01:14:46.853944+0000","last_clean":"2024-05-11T01:14:46.853944+0000","last_became_active":"2024-05-11T01:13:58.435967+0000","last_became_peered":"2024-05-11T01:13:58.435967+0000","last_unstale":"2024-05-11T01:14:46.853944+0000","last_undegraded":"2024-05-11T01:14:46.853944+0000","last_fullsized":"2024-05-11T01:14:46.853944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:22.242555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839175+0000","last_change":"2024-05-11T01:13:59.445506+0000","last_active":"2024-05-11T01:14:46.839175+0000","last_peered":"2024-05-11T01:14:46.839175+0000","last_clean":"2024-05-11T01:14:46.839175+0000","last_became_active":"2024-05-11T01:13:59.445232+0000","last_became_peered":"2024-05-11T01:13:59.445232+0000","last_unstale":"2024-05-11T01:14:46.839175+0000","last_undegraded":"2024-05-11T01:14:46.839175+0000","last_fullsized":"2024-05-11T01:14:46.839175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:18:27.371597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839300+0000","last_change":"2024-05-11T01:13:58.426187+0000","last_active":"2024-05-11T01:14:46.839300+0000","last_peered":"2024-05-11T01:14:46.839300+0000","last_clean":"2024-05-11T01:14:46.839300+0000","last_became_active":"2024-05-11T01:13:58.425913+0000","last_became_peered":"2024-05-11T01:13:58.425913+0000","last_unstale":"2024-05-11T01:14:46.839300+0000","last_undegraded":"2024-05-11T01:14:46.839300+0000","last_fullsized":"2024-05-11T01:14:46.839300+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854582+0000","last_change":"2024-05-11T01:13:59.431401+0000","last_active":"2024-05-11T01:14:46.854582+0000","last_peered":"2024-05-11T01:14:46.854582+0000","last_clean":"2024-05-11T01:14:46.854582+0000","last_became_active":"2024-05-11T01:13:59.431126+0000","last_became_peered":"2024-05-11T01:13:59.431126+0000","last_unstale":"2024-05-11T01:14:46.854582+0000","last_undegraded":"2024-05-11T01:14:46.854582+0000","last_fullsized":"2024-05-11T01:14:46.854582+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:42:47.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855202+0000","last_change":"2024-05-11T01:13:58.431417+0000","last_active":"2024-05-11T01:14:46.855202+0000","last_peered":"2024-05-11T01:14:46.855202+0000","last_clean":"2024-05-11T01:14:46.855202+0000","last_became_active":"2024-05-11T01:13:58.431132+0000","last_became_peered":"2024-05-11T01:13:58.431132+0000","last_unstale":"2024-05-11T01:14:46.855202+0000","last_undegraded":"2024-05-11T01:14:46.855202+0000","last_fullsized":"2024-05-11T01:14:46.855202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:23:16.977274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836973+0000","last_change":"2024-05-11T01:13:59.436396+0000","last_active":"2024-05-11T01:14:46.836973+0000","last_peered":"2024-05-11T01:14:46.836973+0000","last_clean":"2024-05-11T01:14:46.836973+0000","last_became_active":"2024-05-11T01:13:59.436043+0000","last_became_peered":"2024-05-11T01:13:59.436043+0000","last_unstale":"2024-05-11T01:14:46.836973+0000","last_undegraded":"2024-05-11T01:14:46.836973+0000","last_fullsized":"2024-05-11T01:14:46.836973+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854476+0000","last_change":"2024-05-11T01:13:58.435566+0000","last_active":"2024-05-11T01:14:46.854476+0000","last_peered":"2024-05-11T01:14:46.854476+0000","last_clean":"2024-05-11T01:14:46.854476+0000","last_became_active":"2024-05-11T01:13:58.435280+0000","last_became_peered":"2024-05-11T01:13:58.435280+0000","last_unstale":"2024-05-11T01:14:46.854476+0000","last_undegraded":"2024-05-11T01:14:46.854476+0000","last_fullsized":"2024-05-11T01:14:46.854476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:47.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854368+0000","last_change":"2024-05-11T01:13:59.432307+0000","last_active":"2024-05-11T01:14:46.854368+0000","last_peered":"2024-05-11T01:14:46.854368+0000","last_clean":"2024-05-11T01:14:46.854368+0000","last_became_active":"2024-05-11T01:13:59.431913+0000","last_became_peered":"2024-05-11T01:13:59.431913+0000","last_unstale":"2024-05-11T01:14:46.854368+0000","last_undegraded":"2024-05-11T01:14:46.854368+0000","last_fullsized":"2024-05-11T01:14:46.854368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"63'10","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:07.973661+0000","last_change":"2024-05-11T01:13:58.425157+0000","last_active":"2024-05-11T01:16:07.973661+0000","last_peered":"2024-05-11T01:16:07.973661+0000","last_clean":"2024-05-11T01:16:07.973661+0000","last_became_active":"2024-05-11T01:13:58.424894+0000","last_became_peered":"2024-05-11T01:13:58.424894+0000","last_unstale":"2024-05-11T01:16:07.973661+0000","last_undegraded":"2024-05-11T01:16:07.973661+0000","last_fullsized":"2024-05-11T01:16:07.973661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:48:11.424880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837159+0000","last_change":"2024-05-11T01:13:59.447362+0000","last_active":"2024-05-11T01:14:46.837159+0000","last_peered":"2024-05-11T01:14:46.837159+0000","last_clean":"2024-05-11T01:14:46.837159+0000","last_became_active":"2024-05-11T01:13:59.447082+0000","last_became_peered":"2024-05-11T01:13:59.447082+0000","last_unstale":"2024-05-11T01:14:46.837159+0000","last_undegraded":"2024-05-11T01:14:46.837159+0000","last_fullsized":"2024-05-11T01:14:46.837159+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837499+0000","last_change":"2024-05-11T01:13:58.439044+0000","last_active":"2024-05-11T01:14:46.837499+0000","last_peered":"2024-05-11T01:14:46.837499+0000","last_clean":"2024-05-11T01:14:46.837499+0000","last_became_active":"2024-05-11T01:13:58.438603+0000","last_became_peered":"2024-05-11T01:13:58.438603+0000","last_unstale":"2024-05-11T01:14:46.837499+0000","last_undegraded":"2024-05-11T01:14:46.837499+0000","last_fullsized":"2024-05-11T01:14:46.837499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854651+0000","last_change":"2024-05-11T01:13:59.449114+0000","last_active":"2024-05-11T01:14:46.854651+0000","last_peered":"2024-05-11T01:14:46.854651+0000","last_clean":"2024-05-11T01:14:46.854651+0000","last_became_active":"2024-05-11T01:13:59.448532+0000","last_became_peered":"2024-05-11T01:13:59.448532+0000","last_unstale":"2024-05-11T01:14:46.854651+0000","last_undegraded":"2024-05-11T01:14:46.854651+0000","last_fullsized":"2024-05-11T01:14:46.854651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854151+0000","last_change":"2024-05-11T01:13:58.434118+0000","last_active":"2024-05-11T01:14:46.854151+0000","last_peered":"2024-05-11T01:14:46.854151+0000","last_clean":"2024-05-11T01:14:46.854151+0000","last_became_active":"2024-05-11T01:13:58.433818+0000","last_became_peered":"2024-05-11T01:13:58.433818+0000","last_unstale":"2024-05-11T01:14:46.854151+0000","last_undegraded":"2024-05-11T01:14:46.854151+0000","last_fullsized":"2024-05-11T01:14:46.854151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:49.867304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836702+0000","last_change":"2024-05-11T01:13:59.435013+0000","last_active":"2024-05-11T01:14:46.836702+0000","last_peered":"2024-05-11T01:14:46.836702+0000","last_clean":"2024-05-11T01:14:46.836702+0000","last_became_active":"2024-05-11T01:13:59.434726+0000","last_became_peered":"2024-05-11T01:13:59.434726+0000","last_unstale":"2024-05-11T01:14:46.836702+0000","last_undegraded":"2024-05-11T01:14:46.836702+0000","last_fullsized":"2024-05-11T01:14:46.836702+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:03:22.763694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837125+0000","last_change":"2024-05-11T01:13:58.438894+0000","last_active":"2024-05-11T01:14:46.837125+0000","last_peered":"2024-05-11T01:14:46.837125+0000","last_clean":"2024-05-11T01:14:46.837125+0000","last_became_active":"2024-05-11T01:13:58.438528+0000","last_became_peered":"2024-05-11T01:13:58.438528+0000","last_unstale":"2024-05-11T01:14:46.837125+0000","last_undegraded":"2024-05-11T01:14:46.837125+0000","last_fullsized":"2024-05-11T01:14:46.837125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854117+0000","last_change":"2024-05-11T01:13:59.437777+0000","last_active":"2024-05-11T01:14:46.854117+0000","last_peered":"2024-05-11T01:14:46.854117+0000","last_clean":"2024-05-11T01:14:46.854117+0000","last_became_active":"2024-05-11T01:13:59.437176+0000","last_became_peered":"2024-05-11T01:13:59.437176+0000","last_unstale":"2024-05-11T01:14:46.854117+0000","last_undegraded":"2024-05-11T01:14:46.854117+0000","last_fullsized":"2024-05-11T01:14:46.854117+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:07:21.319542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854062+0000","last_change":"2024-05-11T01:13:58.431218+0000","last_active":"2024-05-11T01:14:46.854062+0000","last_peered":"2024-05-11T01:14:46.854062+0000","last_clean":"2024-05-11T01:14:46.854062+0000","last_became_active":"2024-05-11T01:13:58.430952+0000","last_became_peered":"2024-05-11T01:13:58.430952+0000","last_unstale":"2024-05-11T01:14:46.854062+0000","last_undegraded":"2024-05-11T01:14:46.854062+0000","last_fullsized":"2024-05-11T01:14:46.854062+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:03:39.335329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854664+0000","last_change":"2024-05-11T01:13:59.427029+0000","last_active":"2024-05-11T01:14:46.854664+0000","last_peered":"2024-05-11T01:14:46.854664+0000","last_clean":"2024-05-11T01:14:46.854664+0000","last_became_active":"2024-05-11T01:13:59.426727+0000","last_became_peered":"2024-05-11T01:13:59.426727+0000","last_unstale":"2024-05-11T01:14:46.854664+0000","last_undegraded":"2024-05-11T01:14:46.854664+0000","last_fullsized":"2024-05-11T01:14:46.854664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:52.626350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854202+0000","last_change":"2024-05-11T01:13:58.434507+0000","last_active":"2024-05-11T01:14:46.854202+0000","last_peered":"2024-05-11T01:14:46.854202+0000","last_clean":"2024-05-11T01:14:46.854202+0000","last_became_active":"2024-05-11T01:13:58.434215+0000","last_became_peered":"2024-05-11T01:13:58.434215+0000","last_unstale":"2024-05-11T01:14:46.854202+0000","last_undegraded":"2024-05-11T01:14:46.854202+0000","last_fullsized":"2024-05-11T01:14:46.854202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:35.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854789+0000","last_change":"2024-05-11T01:13:59.444388+0000","last_active":"2024-05-11T01:14:46.854789+0000","last_peered":"2024-05-11T01:14:46.854789+0000","last_clean":"2024-05-11T01:14:46.854789+0000","last_became_active":"2024-05-11T01:13:59.443981+0000","last_became_peered":"2024-05-11T01:13:59.443981+0000","last_unstale":"2024-05-11T01:14:46.854789+0000","last_undegraded":"2024-05-11T01:14:46.854789+0000","last_fullsized":"2024-05-11T01:14:46.854789+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839290+0000","last_change":"2024-05-11T01:13:58.429801+0000","last_active":"2024-05-11T01:14:46.839290+0000","last_peered":"2024-05-11T01:14:46.839290+0000","last_clean":"2024-05-11T01:14:46.839290+0000","last_became_active":"2024-05-11T01:13:58.429538+0000","last_became_peered":"2024-05-11T01:13:58.429538+0000","last_unstale":"2024-05-11T01:14:46.839290+0000","last_undegraded":"2024-05-11T01:14:46.839290+0000","last_fullsized":"2024-05-11T01:14:46.839290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:27.314762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837321+0000","last_change":"2024-05-11T01:13:59.438763+0000","last_active":"2024-05-11T01:14:46.837321+0000","last_peered":"2024-05-11T01:14:46.837321+0000","last_clean":"2024-05-11T01:14:46.837321+0000","last_became_active":"2024-05-11T01:13:59.438474+0000","last_became_peered":"2024-05-11T01:13:59.438474+0000","last_unstale":"2024-05-11T01:14:46.837321+0000","last_undegraded":"2024-05-11T01:14:46.837321+0000","last_fullsized":"2024-05-11T01:14:46.837321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854295+0000","last_change":"2024-05-11T01:13:58.426925+0000","last_active":"2024-05-11T01:14:46.854295+0000","last_peered":"2024-05-11T01:14:46.854295+0000","last_clean":"2024-05-11T01:14:46.854295+0000","last_became_active":"2024-05-11T01:13:58.426586+0000","last_became_peered":"2024-05-11T01:13:58.426586+0000","last_unstale":"2024-05-11T01:14:46.854295+0000","last_undegraded":"2024-05-11T01:14:46.854295+0000","last_fullsized":"2024-05-11T01:14:46.854295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837463+0000","last_change":"2024-05-11T01:13:59.436970+0000","last_active":"2024-05-11T01:14:46.837463+0000","last_peered":"2024-05-11T01:14:46.837463+0000","last_clean":"2024-05-11T01:14:46.837463+0000","last_became_active":"2024-05-11T01:13:59.436373+0000","last_became_peered":"2024-05-11T01:13:59.436373+0000","last_unstale":"2024-05-11T01:14:46.837463+0000","last_undegraded":"2024-05-11T01:14:46.837463+0000","last_fullsized":"2024-05-11T01:14:46.837463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837079+0000","last_change":"2024-05-11T01:13:58.439680+0000","last_active":"2024-05-11T01:14:46.837079+0000","last_peered":"2024-05-11T01:14:46.837079+0000","last_clean":"2024-05-11T01:14:46.837079+0000","last_became_active":"2024-05-11T01:13:58.439404+0000","last_became_peered":"2024-05-11T01:13:58.439404+0000","last_unstale":"2024-05-11T01:14:46.837079+0000","last_undegraded":"2024-05-11T01:14:46.837079+0000","last_fullsized":"2024-05-11T01:14:46.837079+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:41:26.236385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837268+0000","last_change":"2024-05-11T01:13:59.435753+0000","last_active":"2024-05-11T01:14:46.837268+0000","last_peered":"2024-05-11T01:14:46.837268+0000","last_clean":"2024-05-11T01:14:46.837268+0000","last_became_active":"2024-05-11T01:13:59.435253+0000","last_became_peered":"2024-05-11T01:13:59.435253+0000","last_unstale":"2024-05-11T01:14:46.837268+0000","last_undegraded":"2024-05-11T01:14:46.837268+0000","last_fullsized":"2024-05-11T01:14:46.837268+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:06:58.758258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837593+0000","last_change":"2024-05-11T01:13:58.437862+0000","last_active":"2024-05-11T01:14:46.837593+0000","last_peered":"2024-05-11T01:14:46.837593+0000","last_clean":"2024-05-11T01:14:46.837593+0000","last_became_active":"2024-05-11T01:13:58.437597+0000","last_became_peered":"2024-05-11T01:13:58.437597+0000","last_unstale":"2024-05-11T01:14:46.837593+0000","last_undegraded":"2024-05-11T01:14:46.837593+0000","last_fullsized":"2024-05-11T01:14:46.837593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:19:23.818943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836437+0000","last_change":"2024-05-11T01:13:59.431673+0000","last_active":"2024-05-11T01:14:46.836437+0000","last_peered":"2024-05-11T01:14:46.836437+0000","last_clean":"2024-05-11T01:14:46.836437+0000","last_became_active":"2024-05-11T01:13:59.431386+0000","last_became_peered":"2024-05-11T01:13:59.431386+0000","last_unstale":"2024-05-11T01:14:46.836437+0000","last_undegraded":"2024-05-11T01:14:46.836437+0000","last_fullsized":"2024-05-11T01:14:46.836437+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:27.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853615+0000","last_change":"2024-05-11T01:13:58.431189+0000","last_active":"2024-05-11T01:14:46.853615+0000","last_peered":"2024-05-11T01:14:46.853615+0000","last_clean":"2024-05-11T01:14:46.853615+0000","last_became_active":"2024-05-11T01:13:58.430890+0000","last_became_peered":"2024-05-11T01:13:58.430890+0000","last_unstale":"2024-05-11T01:14:46.853615+0000","last_undegraded":"2024-05-11T01:14:46.853615+0000","last_fullsized":"2024-05-11T01:14:46.853615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854490+0000","last_change":"2024-05-11T01:13:59.429997+0000","last_active":"2024-05-11T01:14:46.854490+0000","last_peered":"2024-05-11T01:14:46.854490+0000","last_clean":"2024-05-11T01:14:46.854490+0000","last_became_active":"2024-05-11T01:13:59.429734+0000","last_became_peered":"2024-05-11T01:13:59.429734+0000","last_unstale":"2024-05-11T01:14:46.854490+0000","last_undegraded":"2024-05-11T01:14:46.854490+0000","last_fullsized":"2024-05-11T01:14:46.854490+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:47.445191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854269+0000","last_change":"2024-05-11T01:13:58.431136+0000","last_active":"2024-05-11T01:14:46.854269+0000","last_peered":"2024-05-11T01:14:46.854269+0000","last_clean":"2024-05-11T01:14:46.854269+0000","last_became_active":"2024-05-11T01:13:58.430846+0000","last_became_peered":"2024-05-11T01:13:58.430846+0000","last_unstale":"2024-05-11T01:14:46.854269+0000","last_undegraded":"2024-05-11T01:14:46.854269+0000","last_fullsized":"2024-05-11T01:14:46.854269+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:38.439492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854167+0000","last_change":"2024-05-11T01:13:59.450616+0000","last_active":"2024-05-11T01:14:46.854167+0000","last_peered":"2024-05-11T01:14:46.854167+0000","last_clean":"2024-05-11T01:14:46.854167+0000","last_became_active":"2024-05-11T01:13:59.450300+0000","last_became_peered":"2024-05-11T01:13:59.450300+0000","last_unstale":"2024-05-11T01:14:46.854167+0000","last_undegraded":"2024-05-11T01:14:46.854167+0000","last_fullsized":"2024-05-11T01:14:46.854167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854250+0000","last_change":"2024-05-11T01:13:58.436187+0000","last_active":"2024-05-11T01:14:46.854250+0000","last_peered":"2024-05-11T01:14:46.854250+0000","last_clean":"2024-05-11T01:14:46.854250+0000","last_became_active":"2024-05-11T01:13:58.434702+0000","last_became_peered":"2024-05-11T01:13:58.434702+0000","last_unstale":"2024-05-11T01:14:46.854250+0000","last_undegraded":"2024-05-11T01:14:46.854250+0000","last_fullsized":"2024-05-11T01:14:46.854250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:28.737362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837639+0000","last_change":"2024-05-11T01:13:59.452028+0000","last_active":"2024-05-11T01:14:46.837639+0000","last_peered":"2024-05-11T01:14:46.837639+0000","last_clean":"2024-05-11T01:14:46.837639+0000","last_became_active":"2024-05-11T01:13:59.451773+0000","last_became_peered":"2024-05-11T01:13:59.451773+0000","last_unstale":"2024-05-11T01:14:46.837639+0000","last_undegraded":"2024-05-11T01:14:46.837639+0000","last_fullsized":"2024-05-11T01:14:46.837639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"61'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837109+0000","last_change":"2024-05-11T01:13:58.436921+0000","last_active":"2024-05-11T01:14:46.837109+0000","last_peered":"2024-05-11T01:14:46.837109+0000","last_clean":"2024-05-11T01:14:46.837109+0000","last_became_active":"2024-05-11T01:13:58.436650+0000","last_became_peered":"2024-05-11T01:13:58.436650+0000","last_unstale":"2024-05-11T01:14:46.837109+0000","last_undegraded":"2024-05-11T01:14:46.837109+0000","last_fullsized":"2024-05-11T01:14:46.837109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:16:06.268571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854569+0000","last_change":"2024-05-11T01:13:59.442952+0000","last_active":"2024-05-11T01:14:46.854569+0000","last_peered":"2024-05-11T01:14:46.854569+0000","last_clean":"2024-05-11T01:14:46.854569+0000","last_became_active":"2024-05-11T01:13:59.442413+0000","last_became_peered":"2024-05-11T01:13:59.442413+0000","last_unstale":"2024-05-11T01:14:46.854569+0000","last_undegraded":"2024-05-11T01:14:46.854569+0000","last_fullsized":"2024-05-11T01:14:46.854569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:32:54.808774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854522+0000","last_change":"2024-05-11T01:13:58.436342+0000","last_active":"2024-05-11T01:14:46.854522+0000","last_peered":"2024-05-11T01:14:46.854522+0000","last_clean":"2024-05-11T01:14:46.854522+0000","last_became_active":"2024-05-11T01:13:58.436037+0000","last_became_peered":"2024-05-11T01:13:58.436037+0000","last_unstale":"2024-05-11T01:14:46.854522+0000","last_undegraded":"2024-05-11T01:14:46.854522+0000","last_fullsized":"2024-05-11T01:14:46.854522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:22.498662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854431+0000","last_change":"2024-05-11T01:13:59.434239+0000","last_active":"2024-05-11T01:14:46.854431+0000","last_peered":"2024-05-11T01:14:46.854431+0000","last_clean":"2024-05-11T01:14:46.854431+0000","last_became_active":"2024-05-11T01:13:59.433946+0000","last_became_peered":"2024-05-11T01:13:59.433946+0000","last_unstale":"2024-05-11T01:14:46.854431+0000","last_undegraded":"2024-05-11T01:14:46.854431+0000","last_fullsized":"2024-05-11T01:14:46.854431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:07:36.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837338+0000","last_change":"2024-05-11T01:13:58.432458+0000","last_active":"2024-05-11T01:14:46.837338+0000","last_peered":"2024-05-11T01:14:46.837338+0000","last_clean":"2024-05-11T01:14:46.837338+0000","last_became_active":"2024-05-11T01:13:58.432149+0000","last_became_peered":"2024-05-11T01:13:58.432149+0000","last_unstale":"2024-05-11T01:14:46.837338+0000","last_undegraded":"2024-05-11T01:14:46.837338+0000","last_fullsized":"2024-05-11T01:14:46.837338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837389+0000","last_change":"2024-05-11T01:13:59.432345+0000","last_active":"2024-05-11T01:14:46.837389+0000","last_peered":"2024-05-11T01:14:46.837389+0000","last_clean":"2024-05-11T01:14:46.837389+0000","last_became_active":"2024-05-11T01:13:59.432058+0000","last_became_peered":"2024-05-11T01:13:59.432058+0000","last_unstale":"2024-05-11T01:14:46.837389+0000","last_undegraded":"2024-05-11T01:14:46.837389+0000","last_fullsized":"2024-05-11T01:14:46.837389+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:38:32.025100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854602+0000","last_change":"2024-05-11T01:13:58.433877+0000","last_active":"2024-05-11T01:14:46.854602+0000","last_peered":"2024-05-11T01:14:46.854602+0000","last_clean":"2024-05-11T01:14:46.854602+0000","last_became_active":"2024-05-11T01:13:58.433584+0000","last_became_peered":"2024-05-11T01:13:58.433584+0000","last_unstale":"2024-05-11T01:14:46.854602+0000","last_undegraded":"2024-05-11T01:14:46.854602+0000","last_fullsized":"2024-05-11T01:14:46.854602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:20.805314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837457+0000","last_change":"2024-05-11T01:13:59.434011+0000","last_active":"2024-05-11T01:14:46.837457+0000","last_peered":"2024-05-11T01:14:46.837457+0000","last_clean":"2024-05-11T01:14:46.837457+0000","last_became_active":"2024-05-11T01:13:59.432337+0000","last_became_peered":"2024-05-11T01:13:59.432337+0000","last_unstale":"2024-05-11T01:14:46.837457+0000","last_undegraded":"2024-05-11T01:14:46.837457+0000","last_fullsized":"2024-05-11T01:14:46.837457+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839100+0000","last_change":"2024-05-11T01:13:58.435211+0000","last_active":"2024-05-11T01:14:46.839100+0000","last_peered":"2024-05-11T01:14:46.839100+0000","last_clean":"2024-05-11T01:14:46.839100+0000","last_became_active":"2024-05-11T01:13:58.434956+0000","last_became_peered":"2024-05-11T01:13:58.434956+0000","last_unstale":"2024-05-11T01:14:46.839100+0000","last_undegraded":"2024-05-11T01:14:46.839100+0000","last_fullsized":"2024-05-11T01:14:46.839100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853560+0000","last_change":"2024-05-11T01:13:59.432319+0000","last_active":"2024-05-11T01:14:46.853560+0000","last_peered":"2024-05-11T01:14:46.853560+0000","last_clean":"2024-05-11T01:14:46.853560+0000","last_became_active":"2024-05-11T01:13:59.432012+0000","last_became_peered":"2024-05-11T01:13:59.432012+0000","last_unstale":"2024-05-11T01:14:46.853560+0000","last_undegraded":"2024-05-11T01:14:46.853560+0000","last_fullsized":"2024-05-11T01:14:46.853560+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:21.807568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836595+0000","last_change":"2024-05-11T01:13:58.433752+0000","last_active":"2024-05-11T01:14:46.836595+0000","last_peered":"2024-05-11T01:14:46.836595+0000","last_clean":"2024-05-11T01:14:46.836595+0000","last_became_active":"2024-05-11T01:13:58.433411+0000","last_became_peered":"2024-05-11T01:13:58.433411+0000","last_unstale":"2024-05-11T01:14:46.836595+0000","last_undegraded":"2024-05-11T01:14:46.836595+0000","last_fullsized":"2024-05-11T01:14:46.836595+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838906+0000","last_change":"2024-05-11T01:13:59.446971+0000","last_active":"2024-05-11T01:14:46.838906+0000","last_peered":"2024-05-11T01:14:46.838906+0000","last_clean":"2024-05-11T01:14:46.838906+0000","last_became_active":"2024-05-11T01:13:59.446670+0000","last_became_peered":"2024-05-11T01:13:59.446670+0000","last_unstale":"2024-05-11T01:14:46.838906+0000","last_undegraded":"2024-05-11T01:14:46.838906+0000","last_fullsized":"2024-05-11T01:14:46.838906+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:45:46.843918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854241+0000","last_change":"2024-05-11T01:13:58.435876+0000","last_active":"2024-05-11T01:14:46.854241+0000","last_peered":"2024-05-11T01:14:46.854241+0000","last_clean":"2024-05-11T01:14:46.854241+0000","last_became_active":"2024-05-11T01:13:58.434062+0000","last_became_peered":"2024-05-11T01:13:58.434062+0000","last_unstale":"2024-05-11T01:14:46.854241+0000","last_undegraded":"2024-05-11T01:14:46.854241+0000","last_fullsized":"2024-05-11T01:14:46.854241+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:12:26.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855069+0000","last_change":"2024-05-11T01:13:59.445426+0000","last_active":"2024-05-11T01:14:46.855069+0000","last_peered":"2024-05-11T01:14:46.855069+0000","last_clean":"2024-05-11T01:14:46.855069+0000","last_became_active":"2024-05-11T01:13:59.445146+0000","last_became_peered":"2024-05-11T01:13:59.445146+0000","last_unstale":"2024-05-11T01:14:46.855069+0000","last_undegraded":"2024-05-11T01:14:46.855069+0000","last_fullsized":"2024-05-11T01:14:46.855069+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:57:36.879340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854223+0000","last_change":"2024-05-11T01:13:58.434600+0000","last_active":"2024-05-11T01:14:46.854223+0000","last_peered":"2024-05-11T01:14:46.854223+0000","last_clean":"2024-05-11T01:14:46.854223+0000","last_became_active":"2024-05-11T01:13:58.434320+0000","last_became_peered":"2024-05-11T01:13:58.434320+0000","last_unstale":"2024-05-11T01:14:46.854223+0000","last_undegraded":"2024-05-11T01:14:46.854223+0000","last_fullsized":"2024-05-11T01:14:46.854223+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"51'86","reported_seq":246,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853980+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:14:46.853980+0000","last_peered":"2024-05-11T01:14:46.853980+0000","last_clean":"2024-05-11T01:14:46.853980+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:14:46.853980+0000","last_undegraded":"2024-05-11T01:14:46.853980+0000","last_fullsized":"2024-05-11T01:14:46.853980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3807,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":949409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542331,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":757760,"data_stored":359623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54128,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317712,"statfs":{"total":96636764160,"available":96581337088,"internally_reserved":0,"allocated":761856,"data_stored":361960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575033,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44408,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327432,"statfs":{"total":96636764160,"available":96591290368,"internally_reserved":0,"allocated":761856,"data_stored":359589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":765952,"data_stored":361982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58288,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313552,"statfs":{"total":96636764160,"available":96577077248,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575033,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1347584,"data_stored":949353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:16:15.908 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T01:16:16.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.049+0000 7fdc82b80640 1 Processor -- start 2024-05-11T01:16:16.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.050+0000 7fdc82b80640 1 -- start start 2024-05-11T01:16:16.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.050+0000 7fdc82b80640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdc7c0b8a10 0x7fdc7c0d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:16.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.050+0000 7fdc82b80640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 0x7fdc7c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:16.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.050+0000 7fdc82b80640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c0cd6e0 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.050+0000 7fdc82b80640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fdc7c0ce2f0 con 0x7fdc7c0b8a10 2024-05-11T01:16:16.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.050+0000 7fdc82b80640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c0cd9c0 con 0x7fdc7c09c130 2024-05-11T01:16:16.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.050+0000 7fdc8091c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdc7c0b8a10 0x7fdc7c0d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:16.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.050+0000 7fdc8091c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fdc7c0b8a10 0x7fdc7c0d1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43394/0 (socket says 172.21.15.52:43394) 2024-05-11T01:16:16.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.050+0000 7fdc8091c640 1 -- 172.21.15.52:0/2396133788 learned_addr learned my addr 172.21.15.52:0/2396133788 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:16.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.051+0000 7fdc8111d640 1 --2- 172.21.15.52:0/2396133788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 0x7fdc7c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:16.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.051+0000 7fdc8091c640 1 -- 172.21.15.52:0/2396133788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 msgr2=0x7fdc7c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:16.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.051+0000 7fdc8091c640 1 --2- 172.21.15.52:0/2396133788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 0x7fdc7c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:16.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.051+0000 7fdc8091c640 1 -- 172.21.15.52:0/2396133788 >> v1:172.21.15.119:6789/0 conn(0x7fdc7c0b55b0 legacy=0x7fdc7c09ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:16.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.051+0000 7fdc8091c640 1 -- 172.21.15.52:0/2396133788 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c0cdca0 con 0x7fdc7c0b8a10 2024-05-11T01:16:16.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.051+0000 7fdc8091c640 1 --2- 172.21.15.52:0/2396133788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b8a10 0x7fdc7c0d1cf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009e50 tx=0x7fdc7402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2f31b10c13b7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:16.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc7b7fe640 1 -- 172.21.15.52:0/2396133788 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74036070 con 0x7fdc7c0b8a10 2024-05-11T01:16:16.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc7b7fe640 1 -- 172.21.15.52:0/2396133788 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc74031600 con 0x7fdc7c0b8a10 2024-05-11T01:16:16.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc7b7fe640 1 -- 172.21.15.52:0/2396133788 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740319d0 con 0x7fdc7c0b8a10 2024-05-11T01:16:16.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc82b80640 1 -- 172.21.15.52:0/2396133788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b8a10 msgr2=0x7fdc7c0d1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:16.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc82b80640 1 --2- 172.21.15.52:0/2396133788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b8a10 0x7fdc7c0d1cf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009e50 tx=0x7fdc7402a680 comp rx=0 tx=0).stop 2024-05-11T01:16:16.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc82b80640 1 -- 172.21.15.52:0/2396133788 shutdown_connections 2024-05-11T01:16:16.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc82b80640 1 --2- 172.21.15.52:0/2396133788 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 0x7fdc7c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:16.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc82b80640 1 --2- 172.21.15.52:0/2396133788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b8a10 0x7fdc7c0d1cf0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:16.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc82b80640 1 -- 172.21.15.52:0/2396133788 >> 172.21.15.52:0/2396133788 conn(0x7fdc7c0b4a90 msgr2=0x7fdc7c0b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:16.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc82b80640 1 -- 172.21.15.52:0/2396133788 shutdown_connections 2024-05-11T01:16:16.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.052+0000 7fdc82b80640 1 -- 172.21.15.52:0/2396133788 wait complete. 2024-05-11T01:16:16.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.053+0000 7fdc82b80640 1 Processor -- start 2024-05-11T01:16:16.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.053+0000 7fdc82b80640 1 -- start start 2024-05-11T01:16:16.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.054+0000 7fdc82b80640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 0x7fdc7c15f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:16.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.054+0000 7fdc82b80640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b55b0 0x7fdc7c0caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:16.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.054+0000 7fdc82b80640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc7c0b8a10 0x7fdc7c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:16.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.054+0000 7fdc82b80640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c0b5db0 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.054+0000 7fdc82b80640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c0b8df0 con 0x7fdc7c0b8a10 2024-05-11T01:16:16.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.054+0000 7fdc82b80640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c06ee60 con 0x7fdc7c09c130 2024-05-11T01:16:16.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.054+0000 7fdc7bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b55b0 0x7fdc7c0caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:16.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.054+0000 7fdc7bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b55b0 0x7fdc7c0caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43410/0 (socket says 172.21.15.52:43410) 2024-05-11T01:16:16.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.054+0000 7fdc7bfff640 1 -- 172.21.15.52:0/3467988925 learned_addr learned my addr 172.21.15.52:0/3467988925 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:16.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc8111d640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc7c0b8a10 0x7fdc7c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:16.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc8091c640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 0x7fdc7c15f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:16.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc7bfff640 1 -- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 msgr2=0x7fdc7c15f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:16.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc7bfff640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 0x7fdc7c15f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:16.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc7bfff640 1 -- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc7c0b8a10 msgr2=0x7fdc7c0cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:16.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc7bfff640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc7c0b8a10 0x7fdc7c0cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:16.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc7bfff640 1 -- 172.21.15.52:0/3467988925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c0cb300 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc7bfff640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b55b0 0x7fdc7c0caa10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000c3a0 tx=0x7fdc7000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:16.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc797fa640 1 -- 172.21.15.52:0/3467988925 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70017070 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc797fa640 1 -- 172.21.15.52:0/3467988925 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc700052d0 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c0cb5e0 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc797fa640 1 -- 172.21.15.52:0/3467988925 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70003830 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.055+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c0cbba0 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.056+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c069780 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.057+0000 7fdc797fa640 1 -- 172.21.15.52:0/3467988925 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fdc7001a070 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.057+0000 7fdc797fa640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdc680762d0 0x7fdc68078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:16.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.057+0000 7fdc797fa640 1 -- 172.21.15.52:0/3467988925 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7fdc70096bd0 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.057+0000 7fdc8091c640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdc680762d0 0x7fdc68078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:16.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.058+0000 7fdc8091c640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdc680762d0 0x7fdc68078790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc74033000 tx=0x7fdc74009000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:16.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.062+0000 7fdc797fa640 1 -- 172.21.15.52:0/3467988925 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdc700602c0 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.252+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc7c0b4870 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.254+0000 7fdc797fa640 1 -- 172.21.15.52:0/3467988925 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17606 (secure 0 0 0) 0x7fdc700682b0 con 0x7fdc7c0b55b0 2024-05-11T01:16:16.254 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:16:16.254 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":63,"fsid":"f8411cb1-0508-49cc-906f-2364b6835afe","created":"2024-05-11T01:00:45.091136+0000","modified":"2024-05-11T01:14:46.829867+0000","last_up_change":"2024-05-11T01:00:55.632733+0000","last_in_change":"2024-05-11T01:00:47.319302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T01:00:55.525012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-11T01:13:56.478344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-11T01:13:57.758014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffc603c0-0cad-4c06-8325-aafc832216fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6801","nonce":1335322190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6803","nonce":1335322190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6807","nonce":1335322190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1335322190},{"type":"v1","addr":"172.21.15.52:6805","nonce":1335322190}]},"public_addr":"172.21.15.52:6801/1335322190","cluster_addr":"172.21.15.52:6803/1335322190","heartbeat_back_addr":"172.21.15.52:6807/1335322190","heartbeat_front_addr":"172.21.15.52:6805/1335322190","state":["exists","up"]},{"osd":1,"uuid":"ce9d9ccb-7863-4a96-82a3-011cfde3e799","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6809","nonce":1366717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6811","nonce":1366717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6815","nonce":1366717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1366717920},{"type":"v1","addr":"172.21.15.52:6813","nonce":1366717920}]},"public_addr":"172.21.15.52:6809/1366717920","cluster_addr":"172.21.15.52:6811/1366717920","heartbeat_back_addr":"172.21.15.52:6815/1366717920","heartbeat_front_addr":"172.21.15.52:6813/1366717920","state":["exists","up"]},{"osd":2,"uuid":"0f664c50-d358-46c6-b026-ef3106b46648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6825","nonce":410093751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6827","nonce":410093751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6831","nonce":410093751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":410093751},{"type":"v1","addr":"172.21.15.52:6829","nonce":410093751}]},"public_addr":"172.21.15.52:6825/410093751","cluster_addr":"172.21.15.52:6827/410093751","heartbeat_back_addr":"172.21.15.52:6831/410093751","heartbeat_front_addr":"172.21.15.52:6829/410093751","state":["exists","up"]},{"osd":3,"uuid":"26330d08-2e2a-4cdf-bccd-100d145b1e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6817","nonce":2352208085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6819","nonce":2352208085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6823","nonce":2352208085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2352208085},{"type":"v1","addr":"172.21.15.52:6821","nonce":2352208085}]},"public_addr":"172.21.15.52:6817/2352208085","cluster_addr":"172.21.15.52:6819/2352208085","heartbeat_back_addr":"172.21.15.52:6823/2352208085","heartbeat_front_addr":"172.21.15.52:6821/2352208085","state":["exists","up"]},{"osd":4,"uuid":"fcabd274-e244-4a4c-a548-3436c80d0ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6801","nonce":2665075860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6803","nonce":2665075860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6807","nonce":2665075860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2665075860},{"type":"v1","addr":"172.21.15.119:6805","nonce":2665075860}]},"public_addr":"172.21.15.119:6801/2665075860","cluster_addr":"172.21.15.119:6803/2665075860","heartbeat_back_addr":"172.21.15.119:6807/2665075860","heartbeat_front_addr":"172.21.15.119:6805/2665075860","state":["exists","up"]},{"osd":5,"uuid":"e31c2f19-6e3f-4d7d-80fb-aabd637e3bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6825","nonce":3499512602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6827","nonce":3499512602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6831","nonce":3499512602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":3499512602},{"type":"v1","addr":"172.21.15.119:6829","nonce":3499512602}]},"public_addr":"172.21.15.119:6825/3499512602","cluster_addr":"172.21.15.119:6827/3499512602","heartbeat_back_addr":"172.21.15.119:6831/3499512602","heartbeat_front_addr":"172.21.15.119:6829/3499512602","state":["exists","up"]},{"osd":6,"uuid":"d674a25d-afaa-4e54-802f-a352933abb02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6809","nonce":2126488559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6811","nonce":2126488559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6815","nonce":2126488559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2126488559},{"type":"v1","addr":"172.21.15.119:6813","nonce":2126488559}]},"public_addr":"172.21.15.119:6809/2126488559","cluster_addr":"172.21.15.119:6811/2126488559","heartbeat_back_addr":"172.21.15.119:6815/2126488559","heartbeat_front_addr":"172.21.15.119:6813/2126488559","state":["exists","up"]},{"osd":7,"uuid":"3eba8bc2-884e-45a1-b06c-84e62a3300b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6817","nonce":4214692661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6819","nonce":4214692661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6823","nonce":4214692661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":4214692661},{"type":"v1","addr":"172.21.15.119:6821","nonce":4214692661}]},"public_addr":"172.21.15.119:6817/4214692661","cluster_addr":"172.21.15.119:6819/4214692661","heartbeat_back_addr":"172.21.15.119:6823/4214692661","heartbeat_front_addr":"172.21.15.119:6821/4214692661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.386571+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.504914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.468852+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.484405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.379977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:54.589260+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.429532+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T01:00:52.453977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2d","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":9,"length":1},{"begin":13,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":46,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":57,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":67,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":77,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":94,"length":2},{"begin":99,"length":1},{"begin":103,"length":1},{"begin":106,"length":1},{"begin":110,"length":1},{"begin":112,"length":1},{"begin":114,"length":1},{"begin":194,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T01:16:16.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdc680762d0 msgr2=0x7fdc68078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:16.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdc680762d0 0x7fdc68078790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc74033000 tx=0x7fdc74009000 comp rx=0 tx=0).stop 2024-05-11T01:16:16.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b55b0 msgr2=0x7fdc7c0caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:16.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b55b0 0x7fdc7c0caa10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000c3a0 tx=0x7fdc7000c860 comp rx=0 tx=0).stop 2024-05-11T01:16:16.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 shutdown_connections 2024-05-11T01:16:16.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fdc680762d0 0x7fdc68078790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:16.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc7c0b8a10 0x7fdc7c0cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:16.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc7c0b55b0 0x7fdc7c0caa10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:16.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 --2- 172.21.15.52:0/3467988925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc7c09c130 0x7fdc7c15f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:16.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 >> 172.21.15.52:0/3467988925 conn(0x7fdc7c0b4a90 msgr2=0x7fdc7c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:16.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 shutdown_connections 2024-05-11T01:16:16.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:16.256+0000 7fdc82b80640 1 -- 172.21.15.52:0/3467988925 wait complete. 2024-05-11T01:16:17.268 INFO:tasks.ceph:Scrubbing osd.0 2024-05-11T01:16:17.269 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-11T01:16:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.389+0000 7f48e9672640 1 Processor -- start 2024-05-11T01:16:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.389+0000 7f48e9672640 1 -- start start 2024-05-11T01:16:17.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.390+0000 7f48e9672640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f48e415f310 0x7f48e40d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.390+0000 7f48e9672640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f48e40b4dc0 0x7f48e40d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.390+0000 7f48e9672640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48e40d55d0 0x7f48e4162ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.390+0000 7f48e9672640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f48e40b6140 con 0x7f48e40b4dc0 2024-05-11T01:16:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.390+0000 7f48e9672640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f48e40cfd40 con 0x7f48e415f310 2024-05-11T01:16:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.390+0000 7f48e9672640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f48e40d5a00 con 0x7f48e40d55d0 2024-05-11T01:16:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.390+0000 7f48e27fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f48e40b4dc0 0x7f48e40d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.390+0000 7f48e27fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f48e40b4dc0 0x7f48e40d5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43416/0 (socket says 172.21.15.52:43416) 2024-05-11T01:16:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.390+0000 7f48e27fc640 1 -- 172.21.15.52:0/695314139 learned_addr learned my addr 172.21.15.52:0/695314139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:17.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.391+0000 7f48e37fe640 1 --2- 172.21.15.52:0/695314139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48e40d55d0 0x7f48e4162ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.391+0000 7f48e27fc640 1 -- 172.21.15.52:0/695314139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48e40d55d0 msgr2=0x7f48e4162ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:17.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.391+0000 7f48e2ffd640 1 --2- 172.21.15.52:0/695314139 >> v2:172.21.15.119:3300/0 conn(0x7f48e415f310 0x7f48e40d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.391+0000 7f48e27fc640 1 --2- 172.21.15.52:0/695314139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48e40d55d0 0x7f48e4162ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.391+0000 7f48e27fc640 1 -- 172.21.15.52:0/695314139 >> v2:172.21.15.119:3300/0 conn(0x7f48e415f310 msgr2=0x7f48e40d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:17.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.391+0000 7f48e27fc640 1 --2- 172.21.15.52:0/695314139 >> v2:172.21.15.119:3300/0 conn(0x7f48e415f310 0x7f48e40d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.391+0000 7f48e27fc640 1 -- 172.21.15.52:0/695314139 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e40b5530 con 0x7f48e40b4dc0 2024-05-11T01:16:17.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.391+0000 7f48e27fc640 1 --2- 172.21.15.52:0/695314139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 0x7f48e40d5090 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48e408e890 tx=0x7f48d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6e062910f88574 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e1ffb640 1 -- 172.21.15.52:0/695314139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8037070 con 0x7f48e40b4dc0 2024-05-11T01:16:17.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e1ffb640 1 -- 172.21.15.52:0/695314139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d8032440 con 0x7f48e40b4dc0 2024-05-11T01:16:17.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e1ffb640 1 -- 172.21.15.52:0/695314139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8032810 con 0x7f48e40b4dc0 2024-05-11T01:16:17.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e9672640 1 -- 172.21.15.52:0/695314139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 msgr2=0x7f48e40d5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e9672640 1 --2- 172.21.15.52:0/695314139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 0x7f48e40d5090 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48e408e890 tx=0x7f48d802a1e0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e9672640 1 -- 172.21.15.52:0/695314139 shutdown_connections 2024-05-11T01:16:17.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e9672640 1 --2- 172.21.15.52:0/695314139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 0x7f48e40d5090 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e9672640 1 --2- 172.21.15.52:0/695314139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48e40d55d0 0x7f48e4162ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e9672640 1 --2- 172.21.15.52:0/695314139 >> v2:172.21.15.119:3300/0 conn(0x7f48e415f310 0x7f48e40d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e9672640 1 -- 172.21.15.52:0/695314139 >> 172.21.15.52:0/695314139 conn(0x7f48e408eb80 msgr2=0x7f48e409e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:17.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e9672640 1 -- 172.21.15.52:0/695314139 shutdown_connections 2024-05-11T01:16:17.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.392+0000 7f48e9672640 1 -- 172.21.15.52:0/695314139 wait complete. 2024-05-11T01:16:17.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.394+0000 7f48e9672640 1 Processor -- start 2024-05-11T01:16:17.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.394+0000 7f48e9672640 1 -- start start 2024-05-11T01:16:17.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.394+0000 7f48e9672640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 0x7f48e4160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e9672640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48e40d55d0 0x7f48e41ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e9672640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e415f310 0x7f48e41eab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e9672640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f48e40b5dd0 con 0x7f48e40b4dc0 2024-05-11T01:16:17.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e9672640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f48e40b5af0 con 0x7f48e415f310 2024-05-11T01:16:17.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e9672640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f48e40b5250 con 0x7f48e40d55d0 2024-05-11T01:16:17.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 0x7f48e4160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 0x7f48e4160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43418/0 (socket says 172.21.15.52:43418) 2024-05-11T01:16:17.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e2ffd640 1 -- 172.21.15.52:0/3204529191 learned_addr learned my addr 172.21.15.52:0/3204529191 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:17.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e27fc640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48e40d55d0 0x7f48e41ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e37fe640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e415f310 0x7f48e41eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e2ffd640 1 -- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48e40d55d0 msgr2=0x7f48e41ea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e2ffd640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f48e40d55d0 0x7f48e41ea5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e2ffd640 1 -- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e415f310 msgr2=0x7f48e41eab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e2ffd640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e415f310 0x7f48e41eab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e2ffd640 1 -- 172.21.15.52:0/3204529191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e40caf10 con 0x7f48e40b4dc0 2024-05-11T01:16:17.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48e2ffd640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 0x7f48e4160ca0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f48d400c7f0 tx=0x7f48d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.395+0000 7f48cb7fe640 1 -- 172.21.15.52:0/3204529191 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4015070 con 0x7f48e40b4dc0 2024-05-11T01:16:17.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.396+0000 7f48cb7fe640 1 -- 172.21.15.52:0/3204529191 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d4004220 con 0x7f48e40b4dc0 2024-05-11T01:16:17.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.396+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e40cb1f0 con 0x7f48e40b4dc0 2024-05-11T01:16:17.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.396+0000 7f48cb7fe640 1 -- 172.21.15.52:0/3204529191 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4007620 con 0x7f48e40b4dc0 2024-05-11T01:16:17.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.396+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e40cb7b0 con 0x7f48e40b4dc0 2024-05-11T01:16:17.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.396+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48e4069780 con 0x7f48e40b4dc0 2024-05-11T01:16:17.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.397+0000 7f48cb7fe640 1 -- 172.21.15.52:0/3204529191 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f48d40079d0 con 0x7f48e40b4dc0 2024-05-11T01:16:17.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.398+0000 7f48cb7fe640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f48d00762d0 0x7f48d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.398+0000 7f48cb7fe640 1 -- 172.21.15.52:0/3204529191 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f48d4097ce0 con 0x7f48e40b4dc0 2024-05-11T01:16:17.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.398+0000 7f48cb7fe640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7f48d007ea00 0x7f48d0080e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.398+0000 7f48cb7fe640 1 -- 172.21.15.52:0/3204529191 --> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48d400fea0 con 0x7f48d007ea00 2024-05-11T01:16:17.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.398+0000 7f48cb7fe640 1 -- 172.21.15.52:0/3204529191 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f48d4098170 con 0x7f48e40b4dc0 2024-05-11T01:16:17.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.398+0000 7f48e37fe640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7f48d007ea00 0x7f48d0080e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.398+0000 7f48e27fc640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f48d00762d0 0x7f48d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.398+0000 7f48e37fe640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7f48d007ea00 0x7f48d0080e10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.399+0000 7f48e27fc640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f48d00762d0 0x7f48d0078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48d8034000 tx=0x7f48d802e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.399+0000 7f48cb7fe640 1 -- 172.21.15.52:0/3204529191 <== osd.0 v2:172.21.15.52:6800/1335322190 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48d400fea0 con 0x7f48d007ea00 2024-05-11T01:16:17.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 --> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f48e40d1a70 con 0x7f48d007ea00 2024-05-11T01:16:17.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48cb7fe640 1 -- 172.21.15.52:0/3204529191 <== osd.0 v2:172.21.15.52:6800/1335322190 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f48e40d1a70 con 0x7f48d007ea00 2024-05-11T01:16:17.413 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:16:17.413 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T01:16:17.413 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:16:17.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7f48d007ea00 msgr2=0x7f48d0080e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.52:6800/1335322190,v1:172.21.15.52:6801/1335322190] conn(0x7f48d007ea00 0x7f48d0080e10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f48d00762d0 msgr2=0x7f48d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f48d00762d0 0x7f48d0078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48d8034000 tx=0x7f48d802e000 comp rx=0 tx=0).stop 2024-05-11T01:16:17.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 msgr2=0x7f48e4160ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 --2- 172.21.15.52:0/3204529191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48e40b4dc0 0x7f48e4160ca0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f48d400c7f0 tx=0x7f48d400ccb0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e37fe640 1 -- 172.21.15.52:0/3204529191 reap_dead start 2024-05-11T01:16:17.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 shutdown_connections 2024-05-11T01:16:17.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 >> 172.21.15.52:0/3204529191 conn(0x7f48e408eb80 msgr2=0x7f48e406e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:17.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 shutdown_connections 2024-05-11T01:16:17.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.413+0000 7f48e9672640 1 -- 172.21.15.52:0/3204529191 wait complete. 2024-05-11T01:16:17.424 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-11T01:16:17.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.532+0000 7fb392820640 1 Processor -- start 2024-05-11T01:16:17.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.532+0000 7fb392820640 1 -- start start 2024-05-11T01:16:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.532+0000 7fb392820640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb38c15f430 0x7fb38c15fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.532+0000 7fb392820640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb38c1602b0 0x7fb38c0d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.532+0000 7fb392820640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 0x7fb38c0d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.532+0000 7fb392820640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fb38c163250 con 0x7fb38c1602b0 2024-05-11T01:16:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.532+0000 7fb392820640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fb38c163530 con 0x7fb38c15f430 2024-05-11T01:16:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.532+0000 7fb392820640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb38c163810 con 0x7fb38c160720 2024-05-11T01:16:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38b7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb38c1602b0 0x7fb38c0d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38b7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fb38c1602b0 0x7fb38c0d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43434/0 (socket says 172.21.15.52:43434) 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38b7fe640 1 -- 172.21.15.52:0/3945360769 learned_addr learned my addr 172.21.15.52:0/3945360769 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38bfff640 1 --2- 172.21.15.52:0/3945360769 >> v2:172.21.15.119:3300/0 conn(0x7fb38c15f430 0x7fb38c15fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb390dbd640 1 --2- 172.21.15.52:0/3945360769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 0x7fb38c0d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38b7fe640 1 -- 172.21.15.52:0/3945360769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 msgr2=0x7fb38c0d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38b7fe640 1 --2- 172.21.15.52:0/3945360769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 0x7fb38c0d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38b7fe640 1 -- 172.21.15.52:0/3945360769 >> v2:172.21.15.119:3300/0 conn(0x7fb38c15f430 msgr2=0x7fb38c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38b7fe640 1 --2- 172.21.15.52:0/3945360769 >> v2:172.21.15.119:3300/0 conn(0x7fb38c15f430 0x7fb38c15fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38b7fe640 1 -- 172.21.15.52:0/3945360769 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c163af0 con 0x7fb38c1602b0 2024-05-11T01:16:17.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.533+0000 7fb38bfff640 1 --2- 172.21.15.52:0/3945360769 >> v2:172.21.15.119:3300/0 conn(0x7fb38c15f430 0x7fb38c15fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb38b7fe640 1 --2- 172.21.15.52:0/3945360769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1602b0 0x7fb38c0d1b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0b6b80 tx=0x7fb38002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1575c5a178616d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb38affd640 1 -- 172.21.15.52:0/3945360769 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380037070 con 0x7fb38c1602b0 2024-05-11T01:16:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb38affd640 1 -- 172.21.15.52:0/3945360769 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb380032440 con 0x7fb38c1602b0 2024-05-11T01:16:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb38affd640 1 -- 172.21.15.52:0/3945360769 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380032810 con 0x7fb38c1602b0 2024-05-11T01:16:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb392820640 1 -- 172.21.15.52:0/3945360769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1602b0 msgr2=0x7fb38c0d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb392820640 1 --2- 172.21.15.52:0/3945360769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1602b0 0x7fb38c0d1b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0b6b80 tx=0x7fb38002a1e0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb392820640 1 -- 172.21.15.52:0/3945360769 shutdown_connections 2024-05-11T01:16:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb392820640 1 --2- 172.21.15.52:0/3945360769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c1602b0 0x7fb38c0d1b50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb392820640 1 --2- 172.21.15.52:0/3945360769 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 0x7fb38c0d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb392820640 1 --2- 172.21.15.52:0/3945360769 >> v2:172.21.15.119:3300/0 conn(0x7fb38c15f430 0x7fb38c15fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb392820640 1 -- 172.21.15.52:0/3945360769 >> 172.21.15.52:0/3945360769 conn(0x7fb38c08eb70 msgr2=0x7fb38c09e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb392820640 1 -- 172.21.15.52:0/3945360769 shutdown_connections 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.534+0000 7fb392820640 1 -- 172.21.15.52:0/3945360769 wait complete. 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.536+0000 7fb392820640 1 Processor -- start 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.536+0000 7fb392820640 1 -- start start 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.536+0000 7fb392820640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c15f430 0x7fb38c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb392820640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb38c1602b0 0x7fb38c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb392820640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 0x7fb38c0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb392820640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c163030 con 0x7fb38c15f430 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb392820640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c162eb0 con 0x7fb38c1602b0 2024-05-11T01:16:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb392820640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fb38c0d25d0 con 0x7fb38c160720 2024-05-11T01:16:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c15f430 0x7fb38c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c15f430 0x7fb38c15a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43444/0 (socket says 172.21.15.52:43444) 2024-05-11T01:16:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38bfff640 1 -- 172.21.15.52:0/1555273205 learned_addr learned my addr 172.21.15.52:0/1555273205 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38b7fe640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb38c1602b0 0x7fb38c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb390dbd640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 0x7fb38c0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38bfff640 1 -- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 msgr2=0x7fb38c0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38bfff640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 0x7fb38c0ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38bfff640 1 -- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb38c1602b0 msgr2=0x7fb38c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38bfff640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb38c1602b0 0x7fb38c0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38bfff640 1 -- 172.21.15.52:0/1555273205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c0ca8a0 con 0x7fb38c15f430 2024-05-11T01:16:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb390dbd640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 0x7fb38c0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb38bfff640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c15f430 0x7fb38c15a750 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c7c0 tx=0x7fb37c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb388ff9640 1 -- 172.21.15.52:0/1555273205 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c015070 con 0x7fb38c15f430 2024-05-11T01:16:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb388ff9640 1 -- 172.21.15.52:0/1555273205 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37c007a30 con 0x7fb38c15f430 2024-05-11T01:16:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38c0cab80 con 0x7fb38c15f430 2024-05-11T01:16:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb388ff9640 1 -- 172.21.15.52:0/1555273205 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c007e00 con 0x7fb38c15f430 2024-05-11T01:16:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.537+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38c0cb140 con 0x7fb38c15f430 2024-05-11T01:16:17.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.538+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38c069780 con 0x7fb38c15f430 2024-05-11T01:16:17.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.539+0000 7fb388ff9640 1 -- 172.21.15.52:0/1555273205 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fb37c018070 con 0x7fb38c15f430 2024-05-11T01:16:17.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.539+0000 7fb388ff9640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fb378076300 0x7fb3780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.540+0000 7fb388ff9640 1 -- 172.21.15.52:0/1555273205 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7fb37c0976d0 con 0x7fb38c15f430 2024-05-11T01:16:17.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.540+0000 7fb38b7fe640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fb378076300 0x7fb3780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.540+0000 7fb38b7fe640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fb378076300 0x7fb3780787c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb380034000 tx=0x7fb38002e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.544+0000 7fb388ff9640 1 -- 172.21.15.52:0/1555273205 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb37c060770 con 0x7fb38c15f430 2024-05-11T01:16:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.736+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fb38c0b4ba0 con 0x7fb378076300 2024-05-11T01:16:17.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.737+0000 7fb388ff9640 1 -- 172.21.15.52:0/1555273205 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb38c0b4ba0 con 0x7fb378076300 2024-05-11T01:16:17.737 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 0 to deep-scrub 2024-05-11T01:16:17.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fb378076300 msgr2=0x7fb3780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fb378076300 0x7fb3780787c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb380034000 tx=0x7fb38002e000 comp rx=0 tx=0).stop 2024-05-11T01:16:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c15f430 msgr2=0x7fb38c15a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c15f430 0x7fb38c15a750 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c7c0 tx=0x7fb37c00cc80 comp rx=0 tx=0).stop 2024-05-11T01:16:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 shutdown_connections 2024-05-11T01:16:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fb378076300 0x7fb3780787c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb38c160720 0x7fb38c0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb38c1602b0 0x7fb38c0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 --2- 172.21.15.52:0/1555273205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb38c15f430 0x7fb38c15a750 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 >> 172.21.15.52:0/1555273205 conn(0x7fb38c08eb70 msgr2=0x7fb38c15d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 shutdown_connections 2024-05-11T01:16:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.739+0000 7fb392820640 1 -- 172.21.15.52:0/1555273205 wait complete. 2024-05-11T01:16:17.750 INFO:tasks.ceph:Scrubbing osd.1 2024-05-11T01:16:17.751 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-11T01:16:17.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.858+0000 7eff8f5da640 1 Processor -- start 2024-05-11T01:16:17.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.858+0000 7eff8f5da640 1 -- start start 2024-05-11T01:16:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.859+0000 7eff8f5da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff8806d590 0x7eff880cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.859+0000 7eff8f5da640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff88072330 con 0x7eff880cff20 2024-05-11T01:16:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.859+0000 7eff8f5da640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff880d1910 con 0x7eff8815f520 2024-05-11T01:16:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.859+0000 7eff8f5da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7eff880d1bf0 con 0x7eff8806d590 2024-05-11T01:16:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.859+0000 7eff8db77640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7eff880cff20 0x7eff88070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33708/0 (socket says 172.21.15.52:33708) 2024-05-11T01:16:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.859+0000 7eff8db77640 1 -- 172.21.15.52:0/2978854448 learned_addr learned my addr 172.21.15.52:0/2978854448 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.859+0000 7eff8cb75640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff8806d590 0x7eff880cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.859+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 714978277 0 0) 0x7eff88072330 con 0x7eff880cff20 2024-05-11T01:16:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.859+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff74003520 con 0x7eff880cff20 2024-05-11T01:16:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1028986230 0 0) 0x7eff880d1910 con 0x7eff8815f520 2024-05-11T01:16:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff88072330 con 0x7eff8815f520 2024-05-11T01:16:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1728964364 0 0) 0x7eff74003520 con 0x7eff880cff20 2024-05-11T01:16:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff8806d590 msgr2=0x7eff880cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff8806d590 0x7eff880cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 >> v1:172.21.15.119:6789/0 conn(0x7eff8815f520 legacy=0x7eff880d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff880d1ed0 con 0x7eff880cff20 2024-05-11T01:16:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7eff80001a70 con 0x7eff880cff20 2024-05-11T01:16:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 >> v1:172.21.15.52:6789/0 conn(0x7eff880cff20 legacy=0x7eff88070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 0x7eff74004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff74004800 0x7eff74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff740051c0 0x7eff7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff880d1bf0 con 0x7eff74003c60 2024-05-11T01:16:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7eff7400c030 con 0x7eff740051c0 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7eff7400c230 con 0x7eff74004800 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8cb75640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff8806d590 0x7eff880cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8d376640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 0x7eff74004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8db77640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff740051c0 0x7eff7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8cb75640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff74004800 0x7eff74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8d376640 1 -- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff74004800 msgr2=0x7eff74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8d376640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff74004800 0x7eff74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8d376640 1 -- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff740051c0 msgr2=0x7eff7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8d376640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff740051c0 0x7eff7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8d376640 1 -- 172.21.15.52:0/2978854448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff74003520 con 0x7eff74003c60 2024-05-11T01:16:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8cb75640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff74004800 0x7eff74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.860+0000 7eff8db77640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff740051c0 0x7eff7400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.861+0000 7eff8d376640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 0x7eff74004060 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7eff840018a0 tx=0x7eff74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd9dfcaf482502c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.861+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff840014c0 con 0x7eff74003c60 2024-05-11T01:16:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.861+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff8402acf0 con 0x7eff74003c60 2024-05-11T01:16:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.861+0000 7eff7ffff640 1 -- 172.21.15.52:0/2978854448 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84005310 con 0x7eff74003c60 2024-05-11T01:16:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.861+0000 7eff8f5da640 1 -- 172.21.15.52:0/2978854448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 msgr2=0x7eff74004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.861+0000 7eff8f5da640 1 --2- 172.21.15.52:0/2978854448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 0x7eff74004060 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7eff840018a0 tx=0x7eff74005830 comp rx=0 tx=0).stop 2024-05-11T01:16:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.861+0000 7eff8db77640 1 -- 172.21.15.52:0/2978854448 reap_dead start 2024-05-11T01:16:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.862+0000 7eff8f5da640 1 -- 172.21.15.52:0/2978854448 shutdown_connections 2024-05-11T01:16:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.862+0000 7eff8f5da640 1 -- 172.21.15.52:0/2978854448 >> 172.21.15.52:0/2978854448 conn(0x7eff880d3160 msgr2=0x7eff880d3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.862+0000 7eff8f5da640 1 -- 172.21.15.52:0/2978854448 shutdown_connections 2024-05-11T01:16:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.862+0000 7eff8f5da640 1 -- 172.21.15.52:0/2978854448 wait complete. 2024-05-11T01:16:17.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.863+0000 7eff8f5da640 1 Processor -- start 2024-05-11T01:16:17.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.863+0000 7eff8f5da640 1 -- start start 2024-05-11T01:16:17.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.863+0000 7eff8f5da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 0x7eff880b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.863+0000 7eff8f5da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff880b3220 0x7eff880c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.863+0000 7eff8f5da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff880c8310 0x7eff88157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.863+0000 7eff8f5da640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff880cf9e0 con 0x7eff74003c60 2024-05-11T01:16:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.863+0000 7eff8f5da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7eff880d2490 con 0x7eff880b3220 2024-05-11T01:16:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8f5da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7eff880cfb60 con 0x7eff880c8310 2024-05-11T01:16:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8d376640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 0x7eff880b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8d376640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 0x7eff880b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43464/0 (socket says 172.21.15.52:43464) 2024-05-11T01:16:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8d376640 1 -- 172.21.15.52:0/3621370163 learned_addr learned my addr 172.21.15.52:0/3621370163 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:17.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8cb75640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff880b3220 0x7eff880c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8db77640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff880c8310 0x7eff88157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8d376640 1 -- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff880c8310 msgr2=0x7eff88157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8d376640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff880c8310 0x7eff88157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8d376640 1 -- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff880b3220 msgr2=0x7eff880c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8d376640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff880b3220 0x7eff880c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8d376640 1 -- 172.21.15.52:0/3621370163 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff880b1c40 con 0x7eff74003c60 2024-05-11T01:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8cb75640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff880b3220 0x7eff880c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8db77640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff880c8310 0x7eff88157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8d376640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 0x7eff880b2ce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7eff840018a0 tx=0x7eff840012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff7dffb640 1 -- 172.21.15.52:0/3621370163 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff840050d0 con 0x7eff74003c60 2024-05-11T01:16:17.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff7dffb640 1 -- 172.21.15.52:0/3621370163 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff840067d0 con 0x7eff74003c60 2024-05-11T01:16:17.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff880b1f20 con 0x7eff74003c60 2024-05-11T01:16:17.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.864+0000 7eff7dffb640 1 -- 172.21.15.52:0/3621370163 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84006ba0 con 0x7eff74003c60 2024-05-11T01:16:17.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.865+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff880b24e0 con 0x7eff74003c60 2024-05-11T01:16:17.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.865+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff88069780 con 0x7eff74003c60 2024-05-11T01:16:17.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.866+0000 7eff7dffb640 1 -- 172.21.15.52:0/3621370163 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7eff84044060 con 0x7eff74003c60 2024-05-11T01:16:17.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.866+0000 7eff7dffb640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7eff6c0762d0 0x7eff6c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.867+0000 7eff7dffb640 1 -- 172.21.15.52:0/3621370163 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7eff840b6b00 con 0x7eff74003c60 2024-05-11T01:16:17.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.867+0000 7eff7dffb640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7eff6c07ea00 0x7eff6c080e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:17.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.867+0000 7eff7dffb640 1 -- 172.21.15.52:0/3621370163 --> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff84041e80 con 0x7eff6c07ea00 2024-05-11T01:16:17.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.867+0000 7eff7dffb640 1 -- 172.21.15.52:0/3621370163 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7eff840bb2c0 con 0x7eff74003c60 2024-05-11T01:16:17.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.867+0000 7eff8cb75640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7eff6c0762d0 0x7eff6c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.867+0000 7eff8db77640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7eff6c07ea00 0x7eff6c080e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:17.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.867+0000 7eff8db77640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7eff6c07ea00 0x7eff6c080e10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.867+0000 7eff8cb75640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7eff6c0762d0 0x7eff6c078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7eff8815ddf0 tx=0x7eff78008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:17.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.868+0000 7eff7dffb640 1 -- 172.21.15.52:0/3621370163 <== osd.1 v2:172.21.15.52:6808/1366717920 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff84041e80 con 0x7eff6c07ea00 2024-05-11T01:16:17.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.881+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 --> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7eff880c8790 con 0x7eff6c07ea00 2024-05-11T01:16:17.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.881+0000 7eff7dffb640 1 -- 172.21.15.52:0/3621370163 <== osd.1 v2:172.21.15.52:6808/1366717920 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7eff880c8790 con 0x7eff6c07ea00 2024-05-11T01:16:17.881 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:16:17.882 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T01:16:17.882 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:16:17.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.881+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7eff6c07ea00 msgr2=0x7eff6c080e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.881+0000 7eff8f5da640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.52:6808/1366717920,v1:172.21.15.52:6809/1366717920] conn(0x7eff6c07ea00 0x7eff6c080e10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.882+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7eff6c0762d0 msgr2=0x7eff6c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.882+0000 7eff8f5da640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7eff6c0762d0 0x7eff6c078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7eff8815ddf0 tx=0x7eff78008040 comp rx=0 tx=0).stop 2024-05-11T01:16:17.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.882+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 msgr2=0x7eff880b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:17.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.882+0000 7eff8f5da640 1 --2- 172.21.15.52:0/3621370163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74003c60 0x7eff880b2ce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7eff840018a0 tx=0x7eff840012f0 comp rx=0 tx=0).stop 2024-05-11T01:16:17.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.882+0000 7eff8db77640 1 -- 172.21.15.52:0/3621370163 reap_dead start 2024-05-11T01:16:17.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.882+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 shutdown_connections 2024-05-11T01:16:17.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.882+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 >> 172.21.15.52:0/3621370163 conn(0x7eff880d3160 msgr2=0x7eff880b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:17.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.882+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 shutdown_connections 2024-05-11T01:16:17.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.882+0000 7eff8f5da640 1 -- 172.21.15.52:0/3621370163 wait complete. 2024-05-11T01:16:17.892 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-11T01:16:17.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.997+0000 7efe98c08640 1 Processor -- start 2024-05-11T01:16:17.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.997+0000 7efe98c08640 1 -- start start 2024-05-11T01:16:17.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.998+0000 7efe98c08640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe9415f430 0x7efe9415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.998+0000 7efe98c08640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efe941602b0 0x7efe941606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.998+0000 7efe98c08640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe94163250 con 0x7efe940d1b50 2024-05-11T01:16:18.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.998+0000 7efe98c08640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7efe94163530 con 0x7efe941602b0 2024-05-11T01:16:18.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.998+0000 7efe98c08640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efe94163810 con 0x7efe9415f430 2024-05-11T01:16:18.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.998+0000 7efe91d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efe941602b0 0x7efe941606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.998+0000 7efe91d9b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efe941602b0 0x7efe941606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43478/0 (socket says 172.21.15.52:43478) 2024-05-11T01:16:18.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.999+0000 7efe91d9b640 1 -- 172.21.15.52:0/403019378 learned_addr learned my addr 172.21.15.52:0/403019378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:18.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.999+0000 7efe91d9b640 1 -- 172.21.15.52:0/403019378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe9415f430 msgr2=0x7efe9415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:18.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.999+0000 7efe9259c640 1 --2- 172.21.15.52:0/403019378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe9415f430 0x7efe9415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.999+0000 7efe91d9b640 1 --2- 172.21.15.52:0/403019378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe9415f430 0x7efe9415fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.999+0000 7efe91d9b640 1 -- 172.21.15.52:0/403019378 >> v1:172.21.15.119:6789/0 conn(0x7efe940d1b50 legacy=0x7efe940d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.999+0000 7efe91d9b640 1 -- 172.21.15.52:0/403019378 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe94163af0 con 0x7efe941602b0 2024-05-11T01:16:18.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:17.999+0000 7efe9259c640 1 --2- 172.21.15.52:0/403019378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe9415f430 0x7efe9415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:18.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe91d9b640 1 --2- 172.21.15.52:0/403019378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe941602b0 0x7efe941606f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009870 tx=0x7efe7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd82fc72cf623c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe9159a640 1 -- 172.21.15.52:0/403019378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c037070 con 0x7efe941602b0 2024-05-11T01:16:18.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe9159a640 1 -- 172.21.15.52:0/403019378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c032440 con 0x7efe941602b0 2024-05-11T01:16:18.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe9159a640 1 -- 172.21.15.52:0/403019378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c032810 con 0x7efe941602b0 2024-05-11T01:16:18.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe98c08640 1 -- 172.21.15.52:0/403019378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe941602b0 msgr2=0x7efe941606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe98c08640 1 --2- 172.21.15.52:0/403019378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe941602b0 0x7efe941606f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009870 tx=0x7efe7c02a270 comp rx=0 tx=0).stop 2024-05-11T01:16:18.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe98c08640 1 -- 172.21.15.52:0/403019378 shutdown_connections 2024-05-11T01:16:18.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe98c08640 1 --2- 172.21.15.52:0/403019378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe941602b0 0x7efe941606f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe98c08640 1 --2- 172.21.15.52:0/403019378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe9415f430 0x7efe9415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe98c08640 1 -- 172.21.15.52:0/403019378 >> 172.21.15.52:0/403019378 conn(0x7efe9408eb70 msgr2=0x7efe9409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:18.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe98c08640 1 -- 172.21.15.52:0/403019378 shutdown_connections 2024-05-11T01:16:18.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.000+0000 7efe98c08640 1 -- 172.21.15.52:0/403019378 wait complete. 2024-05-11T01:16:18.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.002+0000 7efe98c08640 1 Processor -- start 2024-05-11T01:16:18.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.002+0000 7efe98c08640 1 -- start start 2024-05-11T01:16:18.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.002+0000 7efe98c08640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe940d1b50 0x7efe9415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.002+0000 7efe98c08640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9415f430 0x7efe9415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.002+0000 7efe98c08640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe941602b0 0x7efe940b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.002+0000 7efe98c08640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe94163030 con 0x7efe9415f430 2024-05-11T01:16:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.002+0000 7efe98c08640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe94162eb0 con 0x7efe941602b0 2024-05-11T01:16:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.002+0000 7efe98c08640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efe940d2710 con 0x7efe940d1b50 2024-05-11T01:16:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe91d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9415f430 0x7efe9415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe91d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9415f430 0x7efe9415ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43482/0 (socket says 172.21.15.52:43482) 2024-05-11T01:16:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe91d9b640 1 -- 172.21.15.52:0/973499963 learned_addr learned my addr 172.21.15.52:0/973499963 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe9259c640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe940d1b50 0x7efe9415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe92d9d640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe941602b0 0x7efe940b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe91d9b640 1 -- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe940d1b50 msgr2=0x7efe9415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe91d9b640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe940d1b50 0x7efe9415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe91d9b640 1 -- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe941602b0 msgr2=0x7efe940b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe91d9b640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe941602b0 0x7efe940b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe91d9b640 1 -- 172.21.15.52:0/973499963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9415b600 con 0x7efe9415f430 2024-05-11T01:16:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe9259c640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe940d1b50 0x7efe9415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe91d9b640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9415f430 0x7efe9415ccf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efe7c02a2b0 tx=0x7efe7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe8affd640 1 -- 172.21.15.52:0/973499963 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c037040 con 0x7efe9415f430 2024-05-11T01:16:18.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe8affd640 1 -- 172.21.15.52:0/973499963 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c042070 con 0x7efe9415f430 2024-05-11T01:16:18.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9415b8e0 con 0x7efe9415f430 2024-05-11T01:16:18.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe8affd640 1 -- 172.21.15.52:0/973499963 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c030bb0 con 0x7efe9415f430 2024-05-11T01:16:18.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.003+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe940c9720 con 0x7efe9415f430 2024-05-11T01:16:18.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.004+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe94069780 con 0x7efe9415f430 2024-05-11T01:16:18.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.005+0000 7efe8affd640 1 -- 172.21.15.52:0/973499963 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7efe7c047460 con 0x7efe9415f430 2024-05-11T01:16:18.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.005+0000 7efe8affd640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efe740762d0 0x7efe74078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.006+0000 7efe8affd640 1 -- 172.21.15.52:0/973499963 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7efe7c0b84c0 con 0x7efe9415f430 2024-05-11T01:16:18.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.006+0000 7efe9259c640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efe740762d0 0x7efe74078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.006+0000 7efe9259c640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efe740762d0 0x7efe74078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efe8400d430 tx=0x7efe84007530 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.010+0000 7efe8affd640 1 -- 172.21.15.52:0/973499963 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efe7c081bb0 con 0x7efe9415f430 2024-05-11T01:16:18.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.200+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7efe940cd9a0 con 0x7efe740762d0 2024-05-11T01:16:18.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.200+0000 7efe8affd640 1 -- 172.21.15.52:0/973499963 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efe940cd9a0 con 0x7efe740762d0 2024-05-11T01:16:18.201 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 1 to deep-scrub 2024-05-11T01:16:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efe740762d0 msgr2=0x7efe74078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efe740762d0 0x7efe74078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efe8400d430 tx=0x7efe84007530 comp rx=0 tx=0).stop 2024-05-11T01:16:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9415f430 msgr2=0x7efe9415ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9415f430 0x7efe9415ccf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efe7c02a2b0 tx=0x7efe7c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 shutdown_connections 2024-05-11T01:16:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efe740762d0 0x7efe74078790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe941602b0 0x7efe940b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9415f430 0x7efe9415ccf0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 --2- 172.21.15.52:0/973499963 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efe940d1b50 0x7efe9415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.202+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 >> 172.21.15.52:0/973499963 conn(0x7efe9408eb70 msgr2=0x7efe940d3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.203+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 shutdown_connections 2024-05-11T01:16:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.203+0000 7efe98c08640 1 -- 172.21.15.52:0/973499963 wait complete. 2024-05-11T01:16:18.213 INFO:tasks.ceph:Scrubbing osd.2 2024-05-11T01:16:18.214 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-11T01:16:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.321+0000 7fabf0ed1640 1 Processor -- start 2024-05-11T01:16:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.321+0000 7fabf0ed1640 1 -- start start 2024-05-11T01:16:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabf0ed1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fabec144690 0x7fabec0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabf0ed1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fabec06cb90 0x7fabec144a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabf0ed1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabec163ce0 con 0x7fabec0b4b70 2024-05-11T01:16:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabf0ed1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fabec15f9a0 con 0x7fabec144690 2024-05-11T01:16:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabf0ed1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fabec15fc80 con 0x7fabec06cb90 2024-05-11T01:16:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabea59c640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fabec144690 0x7fabec0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabe9d9b640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fabec0b4b70 0x7fabec06c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33712/0 (socket says 172.21.15.52:33712) 2024-05-11T01:16:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabe9d9b640 1 -- 172.21.15.52:0/3889828235 learned_addr learned my addr 172.21.15.52:0/3889828235 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabead9d640 1 --2- 172.21.15.52:0/3889828235 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fabec06cb90 0x7fabec144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabea59c640 1 -- 172.21.15.52:0/3889828235 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fabec06cb90 msgr2=0x7fabec144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:18.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabea59c640 1 --2- 172.21.15.52:0/3889828235 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fabec06cb90 0x7fabec144a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabea59c640 1 -- 172.21.15.52:0/3889828235 >> v1:172.21.15.52:6789/0 conn(0x7fabec0b4b70 legacy=0x7fabec06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabe959a640 1 -- 172.21.15.52:0/3889828235 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068783284 0 0) 0x7fabec163ce0 con 0x7fabec0b4b70 2024-05-11T01:16:18.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.322+0000 7fabea59c640 1 -- 172.21.15.52:0/3889828235 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabec15ff60 con 0x7fabec144690 2024-05-11T01:16:18.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.323+0000 7fabea59c640 1 --2- 172.21.15.52:0/3889828235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec144690 0x7fabec0b4630 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0097a0 tx=0x7fabdc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd14b049fe3446a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.323+0000 7fabe959a640 1 -- 172.21.15.52:0/3889828235 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc037070 con 0x7fabec144690 2024-05-11T01:16:18.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.323+0000 7fabe959a640 1 -- 172.21.15.52:0/3889828235 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabdc02ae50 con 0x7fabec144690 2024-05-11T01:16:18.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.323+0000 7fabe959a640 1 -- 172.21.15.52:0/3889828235 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc032660 con 0x7fabec144690 2024-05-11T01:16:18.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.324+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3889828235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec144690 msgr2=0x7fabec0b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.324+0000 7fabf0ed1640 1 --2- 172.21.15.52:0/3889828235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec144690 0x7fabec0b4630 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0097a0 tx=0x7fabdc02a100 comp rx=0 tx=0).stop 2024-05-11T01:16:18.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.324+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3889828235 shutdown_connections 2024-05-11T01:16:18.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.324+0000 7fabf0ed1640 1 --2- 172.21.15.52:0/3889828235 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fabec06cb90 0x7fabec144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.324+0000 7fabf0ed1640 1 --2- 172.21.15.52:0/3889828235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec144690 0x7fabec0b4630 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.324+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3889828235 >> 172.21.15.52:0/3889828235 conn(0x7fabec0b3330 msgr2=0x7fabec0b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.324+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3889828235 shutdown_connections 2024-05-11T01:16:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.324+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3889828235 wait complete. 2024-05-11T01:16:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.325+0000 7fabf0ed1640 1 Processor -- start 2024-05-11T01:16:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.325+0000 7fabf0ed1640 1 -- start start 2024-05-11T01:16:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabf0ed1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fabec06cb90 0x7fabec0c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabf0ed1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec0b4b70 0x7fabec0b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabf0ed1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabec144690 0x7fabec15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabf0ed1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fabec0b51a0 con 0x7fabec0b4b70 2024-05-11T01:16:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabf0ed1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fabec160760 con 0x7fabec144690 2024-05-11T01:16:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabf0ed1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fabec0b5320 con 0x7fabec06cb90 2024-05-11T01:16:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabe9d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec0b4b70 0x7fabec0b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabe9d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec0b4b70 0x7fabec0b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43498/0 (socket says 172.21.15.52:43498) 2024-05-11T01:16:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabe9d9b640 1 -- 172.21.15.52:0/3748797459 learned_addr learned my addr 172.21.15.52:0/3748797459 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabea59c640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fabec06cb90 0x7fabec0c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.326+0000 7fabead9d640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabec144690 0x7fabec15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabe9d9b640 1 -- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fabec06cb90 msgr2=0x7fabec0c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabe9d9b640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fabec06cb90 0x7fabec0c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabe9d9b640 1 -- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabec144690 msgr2=0x7fabec15c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabe9d9b640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabec144690 0x7fabec15c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabe9d9b640 1 -- 172.21.15.52:0/3748797459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabec0b1270 con 0x7fabec0b4b70 2024-05-11T01:16:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabe9d9b640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec0b4b70 0x7fabec0b22d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fabe000c490 tx=0x7fabe000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabdaffd640 1 -- 172.21.15.52:0/3748797459 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0018070 con 0x7fabec0b4b70 2024-05-11T01:16:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabdaffd640 1 -- 172.21.15.52:0/3748797459 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe0007500 con 0x7fabec0b4b70 2024-05-11T01:16:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabec0b1510 con 0x7fabec0b4b70 2024-05-11T01:16:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabdaffd640 1 -- 172.21.15.52:0/3748797459 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0004a20 con 0x7fabec0b4b70 2024-05-11T01:16:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.327+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabec0b1ad0 con 0x7fabec0b4b70 2024-05-11T01:16:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.328+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabec069780 con 0x7fabec0b4b70 2024-05-11T01:16:18.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.329+0000 7fabdaffd640 1 -- 172.21.15.52:0/3748797459 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fabe001b070 con 0x7fabec0b4b70 2024-05-11T01:16:18.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.329+0000 7fabdaffd640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fabcc0762d0 0x7fabcc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.329+0000 7fabdaffd640 1 -- 172.21.15.52:0/3748797459 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7fabe0096a80 con 0x7fabec0b4b70 2024-05-11T01:16:18.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.329+0000 7fabea59c640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fabcc0762d0 0x7fabcc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.329+0000 7fabdaffd640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7fabcc07ea00 0x7fabcc080e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.329+0000 7fabdaffd640 1 -- 172.21.15.52:0/3748797459 --> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fabcc081440 con 0x7fabcc07ea00 2024-05-11T01:16:18.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.329+0000 7fabdaffd640 1 -- 172.21.15.52:0/3748797459 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fabe009b2c0 con 0x7fabec0b4b70 2024-05-11T01:16:18.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.329+0000 7fabead9d640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7fabcc07ea00 0x7fabcc080e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.330+0000 7fabead9d640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7fabcc07ea00 0x7fabcc080e10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.330+0000 7fabea59c640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fabcc0762d0 0x7fabcc078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabdc034000 tx=0x7fabdc02e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.330+0000 7fabdaffd640 1 -- 172.21.15.52:0/3748797459 <== osd.2 v2:172.21.15.52:6824/410093751 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabcc081440 con 0x7fabcc07ea00 2024-05-11T01:16:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 --> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fabec06f1a0 con 0x7fabcc07ea00 2024-05-11T01:16:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabdaffd640 1 -- 172.21.15.52:0/3748797459 <== osd.2 v2:172.21.15.52:6824/410093751 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fabec06f1a0 con 0x7fabcc07ea00 2024-05-11T01:16:18.344 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:16:18.344 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T01:16:18.344 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:16:18.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7fabcc07ea00 msgr2=0x7fabcc080e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.52:6824/410093751,v1:172.21.15.52:6825/410093751] conn(0x7fabcc07ea00 0x7fabcc080e10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fabcc0762d0 msgr2=0x7fabcc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fabcc0762d0 0x7fabcc078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabdc034000 tx=0x7fabdc02e000 comp rx=0 tx=0).stop 2024-05-11T01:16:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec0b4b70 msgr2=0x7fabec0b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 --2- 172.21.15.52:0/3748797459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabec0b4b70 0x7fabec0b22d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fabe000c490 tx=0x7fabe000c950 comp rx=0 tx=0).stop 2024-05-11T01:16:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabead9d640 1 -- 172.21.15.52:0/3748797459 reap_dead start 2024-05-11T01:16:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 shutdown_connections 2024-05-11T01:16:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 >> 172.21.15.52:0/3748797459 conn(0x7fabec0b3330 msgr2=0x7fabec143ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:18.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 shutdown_connections 2024-05-11T01:16:18.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.344+0000 7fabf0ed1640 1 -- 172.21.15.52:0/3748797459 wait complete. 2024-05-11T01:16:18.355 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-11T01:16:18.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.460+0000 7f8a69417640 1 Processor -- start 2024-05-11T01:16:18.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.460+0000 7f8a69417640 1 -- start start 2024-05-11T01:16:18.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.460+0000 7f8a69417640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8a640737d0 0x7f8a64166b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.460+0000 7f8a69417640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a640aea90 con 0x7f8a64072ce0 2024-05-11T01:16:18.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.460+0000 7f8a69417640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a640aed70 con 0x7f8a640afb30 2024-05-11T01:16:18.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.460+0000 7f8a69417640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8a6406e760 con 0x7f8a640737d0 2024-05-11T01:16:18.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.461+0000 7f8a637fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8a640737d0 0x7f8a64166b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.461+0000 7f8a62ffd640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8a64072ce0 0x7f8a6406e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33726/0 (socket says 172.21.15.52:33726) 2024-05-11T01:16:18.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.461+0000 7f8a637fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8a640737d0 0x7f8a64166b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43514/0 (socket says 172.21.15.52:43514) 2024-05-11T01:16:18.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.461+0000 7f8a62ffd640 1 -- 172.21.15.52:0/4285758004 learned_addr learned my addr 172.21.15.52:0/4285758004 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:18.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.461+0000 7f8a637fe640 1 -- 172.21.15.52:0/4285758004 >> v1:172.21.15.119:6789/0 conn(0x7f8a640afb30 legacy=0x7f8a640730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.461+0000 7f8a637fe640 1 -- 172.21.15.52:0/4285758004 >> v1:172.21.15.52:6789/0 conn(0x7f8a64072ce0 legacy=0x7f8a6406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.461+0000 7f8a61ffb640 1 -- 172.21.15.52:0/4285758004 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239792935 0 0) 0x7f8a640aea90 con 0x7f8a64072ce0 2024-05-11T01:16:18.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.461+0000 7f8a637fe640 1 -- 172.21.15.52:0/4285758004 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a640af050 con 0x7f8a640737d0 2024-05-11T01:16:18.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a637fe640 1 --2- 172.21.15.52:0/4285758004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640737d0 0x7f8a64166b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009d90 tx=0x7f8a5c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4c06536139eea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a61ffb640 1 -- 172.21.15.52:0/4285758004 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c007740 con 0x7f8a640737d0 2024-05-11T01:16:18.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a61ffb640 1 -- 172.21.15.52:0/4285758004 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5c0078e0 con 0x7f8a640737d0 2024-05-11T01:16:18.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a61ffb640 1 -- 172.21.15.52:0/4285758004 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c007cb0 con 0x7f8a640737d0 2024-05-11T01:16:18.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a69417640 1 -- 172.21.15.52:0/4285758004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640737d0 msgr2=0x7f8a64166b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a69417640 1 --2- 172.21.15.52:0/4285758004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640737d0 0x7f8a64166b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009d90 tx=0x7f8a5c02a6f0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a69417640 1 -- 172.21.15.52:0/4285758004 shutdown_connections 2024-05-11T01:16:18.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a69417640 1 --2- 172.21.15.52:0/4285758004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640737d0 0x7f8a64166b20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a69417640 1 -- 172.21.15.52:0/4285758004 >> 172.21.15.52:0/4285758004 conn(0x7f8a640b2a00 msgr2=0x7f8a640b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:18.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.462+0000 7f8a69417640 1 -- 172.21.15.52:0/4285758004 shutdown_connections 2024-05-11T01:16:18.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.463+0000 7f8a69417640 1 -- 172.21.15.52:0/4285758004 wait complete. 2024-05-11T01:16:18.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.464+0000 7f8a69417640 1 Processor -- start 2024-05-11T01:16:18.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.464+0000 7f8a69417640 1 -- start start 2024-05-11T01:16:18.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a69417640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a64072ce0 0x7f8a641ece10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a69417640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640737d0 0x7f8a641f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a69417640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640afb30 0x7f8a640b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a69417640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a640af610 con 0x7f8a640afb30 2024-05-11T01:16:18.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a69417640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a640b0340 con 0x7f8a640737d0 2024-05-11T01:16:18.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a69417640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8a6409ba20 con 0x7f8a64072ce0 2024-05-11T01:16:18.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a637fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640afb30 0x7f8a640b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a637fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640afb30 0x7f8a640b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43518/0 (socket says 172.21.15.52:43518) 2024-05-11T01:16:18.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a637fe640 1 -- 172.21.15.52:0/800318518 learned_addr learned my addr 172.21.15.52:0/800318518 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:18.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a627fc640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640737d0 0x7f8a641f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a62ffd640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a64072ce0 0x7f8a641ece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a637fe640 1 -- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a64072ce0 msgr2=0x7f8a641ece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a637fe640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a64072ce0 0x7f8a641ece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a637fe640 1 -- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640737d0 msgr2=0x7f8a641f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a637fe640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640737d0 0x7f8a641f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a637fe640 1 -- 172.21.15.52:0/800318518 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a641f3c30 con 0x7f8a640afb30 2024-05-11T01:16:18.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a627fc640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640737d0 0x7f8a641f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:18.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a637fe640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640afb30 0x7f8a640b1540 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009eb0 tx=0x7f8a5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a4b7fe640 1 -- 172.21.15.52:0/800318518 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c040070 con 0x7f8a640afb30 2024-05-11T01:16:18.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a4b7fe640 1 -- 172.21.15.52:0/800318518 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5c031c90 con 0x7f8a640afb30 2024-05-11T01:16:18.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a641539d0 con 0x7f8a640afb30 2024-05-11T01:16:18.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.465+0000 7f8a4b7fe640 1 -- 172.21.15.52:0/800318518 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c03c640 con 0x7f8a640afb30 2024-05-11T01:16:18.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.466+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a64153f90 con 0x7f8a640afb30 2024-05-11T01:16:18.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.466+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a64069780 con 0x7f8a640afb30 2024-05-11T01:16:18.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.467+0000 7f8a4b7fe640 1 -- 172.21.15.52:0/800318518 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f8a5c039040 con 0x7f8a640afb30 2024-05-11T01:16:18.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.468+0000 7f8a4b7fe640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a500762d0 0x7f8a50078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.468+0000 7f8a4b7fe640 1 -- 172.21.15.52:0/800318518 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f8a5c0b8f30 con 0x7f8a640afb30 2024-05-11T01:16:18.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.468+0000 7f8a62ffd640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a500762d0 0x7f8a50078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.469+0000 7f8a62ffd640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a500762d0 0x7f8a50078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000f60 tx=0x7f8a54007b70 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.472+0000 7f8a4b7fe640 1 -- 172.21.15.52:0/800318518 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8a5c082640 con 0x7f8a640afb30 2024-05-11T01:16:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.664+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f8a640b59c0 con 0x7f8a500762d0 2024-05-11T01:16:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.665+0000 7f8a4b7fe640 1 -- 172.21.15.52:0/800318518 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8a640b59c0 con 0x7f8a500762d0 2024-05-11T01:16:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 2 to deep-scrub 2024-05-11T01:16:18.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a500762d0 msgr2=0x7f8a50078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a500762d0 0x7f8a50078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000f60 tx=0x7f8a54007b70 comp rx=0 tx=0).stop 2024-05-11T01:16:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640afb30 msgr2=0x7f8a640b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640afb30 0x7f8a640b1540 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009eb0 tx=0x7f8a5c004a40 comp rx=0 tx=0).stop 2024-05-11T01:16:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 shutdown_connections 2024-05-11T01:16:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8a500762d0 0x7f8a50078790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a640afb30 0x7f8a640b1540 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640737d0 0x7f8a641f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 --2- 172.21.15.52:0/800318518 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8a64072ce0 0x7f8a641ece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 >> 172.21.15.52:0/800318518 conn(0x7f8a640b2a00 msgr2=0x7f8a640b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:18.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 shutdown_connections 2024-05-11T01:16:18.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.667+0000 7f8a69417640 1 -- 172.21.15.52:0/800318518 wait complete. 2024-05-11T01:16:18.679 INFO:tasks.ceph:Scrubbing osd.3 2024-05-11T01:16:18.679 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-11T01:16:18.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.786+0000 7f7dc619e640 1 Processor -- start 2024-05-11T01:16:18.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.786+0000 7f7dc619e640 1 -- start start 2024-05-11T01:16:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.787+0000 7f7dc619e640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7dc00d39b0 0x7f7dc00ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.787+0000 7f7dc619e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7dc00d1390 0x7f7dc00d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.787+0000 7f7dc619e640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc00d1ce0 con 0x7f7dc015d360 2024-05-11T01:16:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.787+0000 7f7dc619e640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc00d0640 con 0x7f7dc00d39b0 2024-05-11T01:16:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.787+0000 7f7dc619e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc00cd8c0 con 0x7f7dc00d1390 2024-05-11T01:16:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.787+0000 7f7dc499b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7dc00d39b0 0x7f7dc00ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.787+0000 7f7dc519c640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f7dc015d360 0x7f7dc00cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33740/0 (socket says 172.21.15.52:33740) 2024-05-11T01:16:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.787+0000 7f7dc519c640 1 -- 172.21.15.52:0/1414766548 learned_addr learned my addr 172.21.15.52:0/1414766548 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.787+0000 7f7dc499b640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7dc00d39b0 0x7f7dc00ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43530/0 (socket says 172.21.15.52:43530) 2024-05-11T01:16:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.788+0000 7f7dc599d640 1 --2- 172.21.15.52:0/1414766548 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7dc00d1390 0x7f7dc00d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.788+0000 7f7dc499b640 1 -- 172.21.15.52:0/1414766548 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7dc00d1390 msgr2=0x7f7dc00d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.788+0000 7f7dc499b640 1 --2- 172.21.15.52:0/1414766548 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7dc00d1390 0x7f7dc00d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.788+0000 7f7dc499b640 1 -- 172.21.15.52:0/1414766548 >> v1:172.21.15.52:6789/0 conn(0x7f7dc015d360 legacy=0x7f7dc00cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.788+0000 7f7dc499b640 1 -- 172.21.15.52:0/1414766548 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc00cdb60 con 0x7f7dc00d39b0 2024-05-11T01:16:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.788+0000 7f7dc499b640 1 --2- 172.21.15.52:0/1414766548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d39b0 0x7f7dc00ce8e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7db40097a0 tx=0x7f7db402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4afab847f901e83 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dbffff640 1 -- 172.21.15.52:0/1414766548 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4037070 con 0x7f7dc00d39b0 2024-05-11T01:16:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dbffff640 1 -- 172.21.15.52:0/1414766548 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db4032440 con 0x7f7dc00d39b0 2024-05-11T01:16:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dbffff640 1 -- 172.21.15.52:0/1414766548 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4032810 con 0x7f7dc00d39b0 2024-05-11T01:16:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dc619e640 1 -- 172.21.15.52:0/1414766548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d39b0 msgr2=0x7f7dc00ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dc619e640 1 --2- 172.21.15.52:0/1414766548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d39b0 0x7f7dc00ce8e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7db40097a0 tx=0x7f7db402a180 comp rx=0 tx=0).stop 2024-05-11T01:16:18.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dc619e640 1 -- 172.21.15.52:0/1414766548 shutdown_connections 2024-05-11T01:16:18.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dc619e640 1 --2- 172.21.15.52:0/1414766548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d39b0 0x7f7dc00ce8e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dc619e640 1 --2- 172.21.15.52:0/1414766548 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7dc00d1390 0x7f7dc00d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dc619e640 1 -- 172.21.15.52:0/1414766548 >> 172.21.15.52:0/1414766548 conn(0x7f7dc00d2db0 msgr2=0x7f7dc00d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:18.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dc619e640 1 -- 172.21.15.52:0/1414766548 shutdown_connections 2024-05-11T01:16:18.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.789+0000 7f7dc619e640 1 -- 172.21.15.52:0/1414766548 wait complete. 2024-05-11T01:16:18.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.791+0000 7f7dc619e640 1 Processor -- start 2024-05-11T01:16:18.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.791+0000 7f7dc619e640 1 -- start start 2024-05-11T01:16:18.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.791+0000 7f7dc619e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d1390 0x7f7dc0151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.791+0000 7f7dc619e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc00d39b0 0x7f7dc0150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.791+0000 7f7dc619e640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7dc015d360 0x7f7dc00cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.791+0000 7f7dc619e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00ce3c0 con 0x7f7dc00d1390 2024-05-11T01:16:18.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.791+0000 7f7dc619e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00ce0e0 con 0x7f7dc00d39b0 2024-05-11T01:16:18.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.791+0000 7f7dc619e640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc00b22c0 con 0x7f7dc015d360 2024-05-11T01:16:18.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.791+0000 7f7dc519c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d1390 0x7f7dc0151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc519c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d1390 0x7f7dc0151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43536/0 (socket says 172.21.15.52:43536) 2024-05-11T01:16:18.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc519c640 1 -- 172.21.15.52:0/1342453750 learned_addr learned my addr 172.21.15.52:0/1342453750 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:18.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc499b640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc00d39b0 0x7f7dc0150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc599d640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7dc015d360 0x7f7dc00cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc519c640 1 -- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7dc015d360 msgr2=0x7f7dc00cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc519c640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7dc015d360 0x7f7dc00cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc519c640 1 -- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc00d39b0 msgr2=0x7f7dc0150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc519c640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc00d39b0 0x7f7dc0150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc519c640 1 -- 172.21.15.52:0/1342453750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc0150e60 con 0x7f7dc00d1390 2024-05-11T01:16:18.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc519c640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d1390 0x7f7dc0151e10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c400 tx=0x7f7db800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dbdffb640 1 -- 172.21.15.52:0/1342453750 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8017070 con 0x7f7dc00d1390 2024-05-11T01:16:18.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dbdffb640 1 -- 172.21.15.52:0/1342453750 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db8005d00 con 0x7f7dc00d1390 2024-05-11T01:16:18.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc0151100 con 0x7f7dc00d1390 2024-05-11T01:16:18.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dbdffb640 1 -- 172.21.15.52:0/1342453750 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8007310 con 0x7f7dc00d1390 2024-05-11T01:16:18.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.792+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc015dd90 con 0x7f7dc00d1390 2024-05-11T01:16:18.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.793+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7dc00d17a0 con 0x7f7dc00d1390 2024-05-11T01:16:18.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.794+0000 7f7dbdffb640 1 -- 172.21.15.52:0/1342453750 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f7db801a070 con 0x7f7dc00d1390 2024-05-11T01:16:18.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.795+0000 7f7dbdffb640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7dac0762d0 0x7f7dac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.795+0000 7f7dbdffb640 1 -- 172.21.15.52:0/1342453750 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f7db8096bd0 con 0x7f7dc00d1390 2024-05-11T01:16:18.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.795+0000 7f7dbdffb640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7f7dac07ea00 0x7f7dac080e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.795+0000 7f7dbdffb640 1 -- 172.21.15.52:0/1342453750 --> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7dac081440 con 0x7f7dac07ea00 2024-05-11T01:16:18.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.795+0000 7f7dbdffb640 1 -- 172.21.15.52:0/1342453750 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f7db809c2a0 con 0x7f7dc00d1390 2024-05-11T01:16:18.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.795+0000 7f7dc499b640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7dac0762d0 0x7f7dac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.795+0000 7f7dc599d640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7f7dac07ea00 0x7f7dac080e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.795+0000 7f7dc599d640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7f7dac07ea00 0x7f7dac080e10 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.795+0000 7f7dc499b640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7dac0762d0 0x7f7dac078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7db4034000 tx=0x7f7db402e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.796+0000 7f7dbdffb640 1 -- 172.21.15.52:0/1342453750 <== osd.3 v2:172.21.15.52:6816/2352208085 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7dac081440 con 0x7f7dac07ea00 2024-05-11T01:16:18.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.809+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 --> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7dc00695f0 con 0x7f7dac07ea00 2024-05-11T01:16:18.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.809+0000 7f7dbdffb640 1 -- 172.21.15.52:0/1342453750 <== osd.3 v2:172.21.15.52:6816/2352208085 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7dc00695f0 con 0x7f7dac07ea00 2024-05-11T01:16:18.810 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:16:18.810 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T01:16:18.810 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:16:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7f7dac07ea00 msgr2=0x7f7dac080e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.52:6816/2352208085,v1:172.21.15.52:6817/2352208085] conn(0x7f7dac07ea00 0x7f7dac080e10 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7dac0762d0 msgr2=0x7f7dac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7dac0762d0 0x7f7dac078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7db4034000 tx=0x7f7db402e000 comp rx=0 tx=0).stop 2024-05-11T01:16:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d1390 msgr2=0x7f7dc0151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 --2- 172.21.15.52:0/1342453750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dc00d1390 0x7f7dc0151e10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c400 tx=0x7f7db800c8c0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc599d640 1 -- 172.21.15.52:0/1342453750 reap_dead start 2024-05-11T01:16:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 shutdown_connections 2024-05-11T01:16:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 >> 172.21.15.52:0/1342453750 conn(0x7f7dc00d2db0 msgr2=0x7f7dc00b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 shutdown_connections 2024-05-11T01:16:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.810+0000 7f7dc619e640 1 -- 172.21.15.52:0/1342453750 wait complete. 2024-05-11T01:16:18.821 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-11T01:16:18.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.925+0000 7f8125bc1640 1 Processor -- start 2024-05-11T01:16:18.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.925+0000 7f8125bc1640 1 -- start start 2024-05-11T01:16:18.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.926+0000 7f8125bc1640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f812015f430 0x7f812015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.926+0000 7f8125bc1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81201602b0 0x7f81200d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.926+0000 7f8125bc1640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8120163250 con 0x7f8120160720 2024-05-11T01:16:18.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.926+0000 7f8125bc1640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8120163530 con 0x7f812015f430 2024-05-11T01:16:18.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.926+0000 7f8125bc1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8120163810 con 0x7f81201602b0 2024-05-11T01:16:18.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.926+0000 7f811f7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f812015f430 0x7f812015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.926+0000 7f811ffff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f8120160720 0x7f81200d2090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33746/0 (socket says 172.21.15.52:33746) 2024-05-11T01:16:18.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.926+0000 7f811ffff640 1 -- 172.21.15.52:0/850463411 learned_addr learned my addr 172.21.15.52:0/850463411 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:18.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.927+0000 7f811effd640 1 --2- 172.21.15.52:0/850463411 >> v2:172.21.15.119:3300/0 conn(0x7f81201602b0 0x7f81200d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.927+0000 7f811f7fe640 1 -- 172.21.15.52:0/850463411 >> v2:172.21.15.119:3300/0 conn(0x7f81201602b0 msgr2=0x7f81200d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:18.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.927+0000 7f811f7fe640 1 --2- 172.21.15.52:0/850463411 >> v2:172.21.15.119:3300/0 conn(0x7f81201602b0 0x7f81200d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.927+0000 7f811f7fe640 1 -- 172.21.15.52:0/850463411 >> v1:172.21.15.52:6789/0 conn(0x7f8120160720 legacy=0x7f81200d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.927+0000 7f811f7fe640 1 -- 172.21.15.52:0/850463411 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8120163af0 con 0x7f812015f430 2024-05-11T01:16:18.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.927+0000 7f811e7fc640 1 -- 172.21.15.52:0/850463411 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141748094 0 0) 0x7f8120163250 con 0x7f8120160720 2024-05-11T01:16:18.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.927+0000 7f811effd640 1 --2- 172.21.15.52:0/850463411 >> v2:172.21.15.119:3300/0 conn(0x7f81201602b0 0x7f81200d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:18.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.927+0000 7f811f7fe640 1 --2- 172.21.15.52:0/850463411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f812015f430 0x7f812015fd70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f81200b4840 tx=0x7f811002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a6dd67ad8a4f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f811e7fc640 1 -- 172.21.15.52:0/850463411 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110037070 con 0x7f812015f430 2024-05-11T01:16:18.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f811e7fc640 1 -- 172.21.15.52:0/850463411 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8110032440 con 0x7f812015f430 2024-05-11T01:16:18.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f811e7fc640 1 -- 172.21.15.52:0/850463411 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811002ae60 con 0x7f812015f430 2024-05-11T01:16:18.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f8125bc1640 1 -- 172.21.15.52:0/850463411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f812015f430 msgr2=0x7f812015fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f8125bc1640 1 --2- 172.21.15.52:0/850463411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f812015f430 0x7f812015fd70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f81200b4840 tx=0x7f811002a110 comp rx=0 tx=0).stop 2024-05-11T01:16:18.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f8125bc1640 1 -- 172.21.15.52:0/850463411 shutdown_connections 2024-05-11T01:16:18.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f8125bc1640 1 --2- 172.21.15.52:0/850463411 >> v2:172.21.15.119:3300/0 conn(0x7f81201602b0 0x7f81200d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f8125bc1640 1 --2- 172.21.15.52:0/850463411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f812015f430 0x7f812015fd70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f8125bc1640 1 -- 172.21.15.52:0/850463411 >> 172.21.15.52:0/850463411 conn(0x7f812008eb70 msgr2=0x7f812009e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:18.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f8125bc1640 1 -- 172.21.15.52:0/850463411 shutdown_connections 2024-05-11T01:16:18.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.928+0000 7f8125bc1640 1 -- 172.21.15.52:0/850463411 wait complete. 2024-05-11T01:16:18.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.929+0000 7f8125bc1640 1 Processor -- start 2024-05-11T01:16:18.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f8125bc1640 1 -- start start 2024-05-11T01:16:18.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f8125bc1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f812015f430 0x7f8120150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f8125bc1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f81201602b0 0x7f812015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f8125bc1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8120160720 0x7f812015c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f8125bc1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8120163030 con 0x7f8120160720 2024-05-11T01:16:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f8125bc1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8120162eb0 con 0x7f812015f430 2024-05-11T01:16:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f8125bc1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f81200d27a0 con 0x7f81201602b0 2024-05-11T01:16:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f811ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8120160720 0x7f812015c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f811ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8120160720 0x7f812015c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43556/0 (socket says 172.21.15.52:43556) 2024-05-11T01:16:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.930+0000 7f811ffff640 1 -- 172.21.15.52:0/3869758715 learned_addr learned my addr 172.21.15.52:0/3869758715 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f811f7fe640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f812015f430 0x7f8120150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f811effd640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f81201602b0 0x7f812015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f811ffff640 1 -- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f81201602b0 msgr2=0x7f812015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f811ffff640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f81201602b0 0x7f812015bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f811ffff640 1 -- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f812015f430 msgr2=0x7f8120150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f811ffff640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f812015f430 0x7f8120150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f811ffff640 1 -- 172.21.15.52:0/3869758715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812015aad0 con 0x7f8120160720 2024-05-11T01:16:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f811f7fe640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f812015f430 0x7f8120150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f811ffff640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8120160720 0x7f812015c400 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f811800c6e0 tx=0x7f811800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f8107fff640 1 -- 172.21.15.52:0/3869758715 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118018070 con 0x7f8120160720 2024-05-11T01:16:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f8107fff640 1 -- 172.21.15.52:0/3869758715 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8118004d60 con 0x7f8120160720 2024-05-11T01:16:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f8107fff640 1 -- 172.21.15.52:0/3869758715 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811800a880 con 0x7f8120160720 2024-05-11T01:16:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81200b2040 con 0x7f8120160720 2024-05-11T01:16:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.931+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81200b2b20 con 0x7f8120160720 2024-05-11T01:16:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.932+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8120069780 con 0x7f8120160720 2024-05-11T01:16:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.933+0000 7f8107fff640 1 -- 172.21.15.52:0/3869758715 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f811801b070 con 0x7f8120160720 2024-05-11T01:16:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.933+0000 7f8107fff640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f810c076300 0x7f810c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.933+0000 7f8107fff640 1 -- 172.21.15.52:0/3869758715 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f8118096d40 con 0x7f8120160720 2024-05-11T01:16:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.934+0000 7f811f7fe640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f810c076300 0x7f810c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.934+0000 7f811f7fe640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f810c076300 0x7f810c0787c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8110034000 tx=0x7f811002e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:18.938+0000 7f8107fff640 1 -- 172.21.15.52:0/3869758715 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8118060430 con 0x7f8120160720 2024-05-11T01:16:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.129+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f81200b4ba0 con 0x7f810c076300 2024-05-11T01:16:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.130+0000 7f8107fff640 1 -- 172.21.15.52:0/3869758715 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f81200b4ba0 con 0x7f810c076300 2024-05-11T01:16:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 3 to deep-scrub 2024-05-11T01:16:19.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f810c076300 msgr2=0x7f810c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f810c076300 0x7f810c0787c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8110034000 tx=0x7f811002e000 comp rx=0 tx=0).stop 2024-05-11T01:16:19.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8120160720 msgr2=0x7f812015c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8120160720 0x7f812015c400 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f811800c6e0 tx=0x7f811800cba0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 shutdown_connections 2024-05-11T01:16:19.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f810c076300 0x7f810c0787c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8120160720 0x7f812015c400 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f81201602b0 0x7f812015bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 --2- 172.21.15.52:0/3869758715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f812015f430 0x7f8120150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 >> 172.21.15.52:0/3869758715 conn(0x7f812008eb70 msgr2=0x7f812006d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:19.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 shutdown_connections 2024-05-11T01:16:19.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.132+0000 7f8125bc1640 1 -- 172.21.15.52:0/3869758715 wait complete. 2024-05-11T01:16:19.144 INFO:tasks.ceph:Scrubbing osd.4 2024-05-11T01:16:19.144 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-11T01:16:19.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.252+0000 7f7c53366640 1 Processor -- start 2024-05-11T01:16:19.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.252+0000 7f7c53366640 1 -- start start 2024-05-11T01:16:19.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c53366640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7c4c15f520 0x7f7c4c0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c53366640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c4c0d00f0 0x7f7c4c070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c53366640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c4c072330 con 0x7f7c4c06d590 2024-05-11T01:16:19.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c53366640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f7c4c0d1670 con 0x7f7c4c15f520 2024-05-11T01:16:19.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c53366640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f7c4c0d1910 con 0x7f7c4c0d00f0 2024-05-11T01:16:19.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c51102640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f7c4c15f520 0x7f7c4c0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c50901640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f7c4c06d590 0x7f7c4c0cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33758/0 (socket says 172.21.15.52:33758) 2024-05-11T01:16:19.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c50901640 1 -- 172.21.15.52:0/3862731030 learned_addr learned my addr 172.21.15.52:0/3862731030 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:19.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c51102640 1 -- 172.21.15.52:0/3862731030 >> v2:172.21.15.119:3300/0 conn(0x7f7c4c0d00f0 msgr2=0x7f7c4c070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:19.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c51903640 1 --2- 172.21.15.52:0/3862731030 >> v2:172.21.15.119:3300/0 conn(0x7f7c4c0d00f0 0x7f7c4c070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c51102640 1 --2- 172.21.15.52:0/3862731030 >> v2:172.21.15.119:3300/0 conn(0x7f7c4c0d00f0 0x7f7c4c070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c43fff640 1 -- 172.21.15.52:0/3862731030 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560992970 0 0) 0x7f7c4c072330 con 0x7f7c4c06d590 2024-05-11T01:16:19.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c51102640 1 -- 172.21.15.52:0/3862731030 >> v1:172.21.15.52:6789/0 conn(0x7f7c4c06d590 legacy=0x7f7c4c0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c51102640 1 -- 172.21.15.52:0/3862731030 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c0d1bf0 con 0x7f7c4c15f520 2024-05-11T01:16:19.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.253+0000 7f7c51903640 1 --2- 172.21.15.52:0/3862731030 >> v2:172.21.15.119:3300/0 conn(0x7f7c4c0d00f0 0x7f7c4c070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:19.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.254+0000 7f7c51102640 1 --2- 172.21.15.52:0/3862731030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c15f520 0x7f7c4c0d1130 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7c480097a0 tx=0x7f7c4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfcd77c795b5dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.254+0000 7f7c43fff640 1 -- 172.21.15.52:0/3862731030 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48037070 con 0x7f7c4c15f520 2024-05-11T01:16:19.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.254+0000 7f7c43fff640 1 -- 172.21.15.52:0/3862731030 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c48032440 con 0x7f7c4c15f520 2024-05-11T01:16:19.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.254+0000 7f7c43fff640 1 -- 172.21.15.52:0/3862731030 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48032810 con 0x7f7c4c15f520 2024-05-11T01:16:19.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.255+0000 7f7c53366640 1 -- 172.21.15.52:0/3862731030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c15f520 msgr2=0x7f7c4c0d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.255+0000 7f7c53366640 1 --2- 172.21.15.52:0/3862731030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c15f520 0x7f7c4c0d1130 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7c480097a0 tx=0x7f7c4802a270 comp rx=0 tx=0).stop 2024-05-11T01:16:19.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.255+0000 7f7c53366640 1 -- 172.21.15.52:0/3862731030 shutdown_connections 2024-05-11T01:16:19.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.255+0000 7f7c53366640 1 --2- 172.21.15.52:0/3862731030 >> v2:172.21.15.119:3300/0 conn(0x7f7c4c0d00f0 0x7f7c4c070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.255+0000 7f7c53366640 1 --2- 172.21.15.52:0/3862731030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c15f520 0x7f7c4c0d1130 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.255+0000 7f7c53366640 1 -- 172.21.15.52:0/3862731030 >> 172.21.15.52:0/3862731030 conn(0x7f7c4c0d3160 msgr2=0x7f7c4c0d3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:19.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.255+0000 7f7c53366640 1 -- 172.21.15.52:0/3862731030 shutdown_connections 2024-05-11T01:16:19.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.255+0000 7f7c53366640 1 -- 172.21.15.52:0/3862731030 wait complete. 2024-05-11T01:16:19.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.256+0000 7f7c53366640 1 Processor -- start 2024-05-11T01:16:19.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.256+0000 7f7c53366640 1 -- start start 2024-05-11T01:16:19.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c53366640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06d590 0x7f7c4c1e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c53366640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c0d00f0 0x7f7c4c09eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c53366640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7c4c15f520 0x7f7c4c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c53366640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c0d2490 con 0x7f7c4c06d590 2024-05-11T01:16:19.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c53366640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c0d21b0 con 0x7f7c4c0d00f0 2024-05-11T01:16:19.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c53366640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4c070c40 con 0x7f7c4c15f520 2024-05-11T01:16:19.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c51102640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06d590 0x7f7c4c1e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c51102640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06d590 0x7f7c4c1e79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43562/0 (socket says 172.21.15.52:43562) 2024-05-11T01:16:19.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c51102640 1 -- 172.21.15.52:0/3121327268 learned_addr learned my addr 172.21.15.52:0/3121327268 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c51102640 1 -- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7c4c15f520 msgr2=0x7f7c4c15e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c51102640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7c4c15f520 0x7f7c4c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c51102640 1 -- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c0d00f0 msgr2=0x7f7c4c09eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c51102640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c4c0d00f0 0x7f7c4c09eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c51102640 1 -- 172.21.15.52:0/3121327268 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c15e7a0 con 0x7f7c4c06d590 2024-05-11T01:16:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.257+0000 7f7c51102640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06d590 0x7f7c4c1e79f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7c48000c00 tx=0x7f7c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.258+0000 7f7c41ffb640 1 -- 172.21.15.52:0/3121327268 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48037040 con 0x7f7c4c06d590 2024-05-11T01:16:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.258+0000 7f7c41ffb640 1 -- 172.21.15.52:0/3121327268 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c48042070 con 0x7f7c4c06d590 2024-05-11T01:16:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.258+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c09f1a0 con 0x7f7c4c06d590 2024-05-11T01:16:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.258+0000 7f7c41ffb640 1 -- 172.21.15.52:0/3121327268 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48030a70 con 0x7f7c4c06d590 2024-05-11T01:16:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.258+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c09f760 con 0x7f7c4c06d590 2024-05-11T01:16:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.259+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c4c09fdb0 con 0x7f7c4c06d590 2024-05-11T01:16:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.259+0000 7f7c41ffb640 1 -- 172.21.15.52:0/3121327268 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f7c48030c10 con 0x7f7c4c06d590 2024-05-11T01:16:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.260+0000 7f7c41ffb640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7c300762d0 0x7f7c30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.260+0000 7f7c41ffb640 1 -- 172.21.15.52:0/3121327268 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f7c480b8cc0 con 0x7f7c4c06d590 2024-05-11T01:16:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.260+0000 7f7c41ffb640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7f7c3007ea00 0x7f7c30080e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.260+0000 7f7c41ffb640 1 -- 172.21.15.52:0/3121327268 --> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c4804ae80 con 0x7f7c3007ea00 2024-05-11T01:16:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.260+0000 7f7c41ffb640 1 -- 172.21.15.52:0/3121327268 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f7c480b9150 con 0x7f7c4c06d590 2024-05-11T01:16:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.260+0000 7f7c50901640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7c300762d0 0x7f7c30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.260+0000 7f7c51903640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7f7c3007ea00 0x7f7c30080e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.261+0000 7f7c50901640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7c300762d0 0x7f7c30078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c15eda0 tx=0x7f7c3c0078d0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.261+0000 7f7c51903640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7f7c3007ea00 0x7f7c30080e10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.262+0000 7f7c41ffb640 1 -- 172.21.15.52:0/3121327268 <== osd.4 v2:172.21.15.119:6800/2665075860 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c4804ae80 con 0x7f7c3007ea00 2024-05-11T01:16:19.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.275+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 --> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7c4c069780 con 0x7f7c3007ea00 2024-05-11T01:16:19.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.275+0000 7f7c41ffb640 1 -- 172.21.15.52:0/3121327268 <== osd.4 v2:172.21.15.119:6800/2665075860 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7c4c069780 con 0x7f7c3007ea00 2024-05-11T01:16:19.276 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:16:19.276 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T01:16:19.276 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:16:19.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7f7c3007ea00 msgr2=0x7f7c30080e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6800/2665075860,v1:172.21.15.119:6801/2665075860] conn(0x7f7c3007ea00 0x7f7c30080e10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7c300762d0 msgr2=0x7f7c30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f7c300762d0 0x7f7c30078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c15eda0 tx=0x7f7c3c0078d0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06d590 msgr2=0x7f7c4c1e79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 --2- 172.21.15.52:0/3121327268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c4c06d590 0x7f7c4c1e79f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7c48000c00 tx=0x7f7c480046c0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c51903640 1 -- 172.21.15.52:0/3121327268 reap_dead start 2024-05-11T01:16:19.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 shutdown_connections 2024-05-11T01:16:19.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 >> 172.21.15.52:0/3121327268 conn(0x7f7c4c0d3160 msgr2=0x7f7c4c06e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:19.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 shutdown_connections 2024-05-11T01:16:19.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.276+0000 7f7c53366640 1 -- 172.21.15.52:0/3121327268 wait complete. 2024-05-11T01:16:19.287 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-11T01:16:19.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.392+0000 7faa46054640 1 Processor -- start 2024-05-11T01:16:19.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.392+0000 7faa46054640 1 -- start start 2024-05-11T01:16:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa46054640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4015f2d0 0x7faa4006e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa46054640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7faa400d35b0 0x7faa400b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa46054640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa400d1f90 con 0x7faa4006e5d0 2024-05-11T01:16:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa46054640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7faa400d2270 con 0x7faa400d35b0 2024-05-11T01:16:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa46054640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7faa400d2550 con 0x7faa4015f2d0 2024-05-11T01:16:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa3ffff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7faa400d35b0 0x7faa400b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa37fff640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7faa4006e5d0 0x7faa400d2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33772/0 (socket says 172.21.15.52:33772) 2024-05-11T01:16:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa37fff640 1 -- 172.21.15.52:0/542996696 learned_addr learned my addr 172.21.15.52:0/542996696 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa3f7fe640 1 --2- 172.21.15.52:0/542996696 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4015f2d0 0x7faa4006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa3ffff640 1 -- 172.21.15.52:0/542996696 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4015f2d0 msgr2=0x7faa4006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa3ffff640 1 --2- 172.21.15.52:0/542996696 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4015f2d0 0x7faa4006e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa3ffff640 1 -- 172.21.15.52:0/542996696 >> v1:172.21.15.52:6789/0 conn(0x7faa4006e5d0 legacy=0x7faa400d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa3ffff640 1 -- 172.21.15.52:0/542996696 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa400d2830 con 0x7faa400d35b0 2024-05-11T01:16:19.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.393+0000 7faa3effd640 1 -- 172.21.15.52:0/542996696 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112186478 0 0) 0x7faa400d1f90 con 0x7faa4006e5d0 2024-05-11T01:16:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.394+0000 7faa3ffff640 1 --2- 172.21.15.52:0/542996696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa400d35b0 0x7faa400b2600 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faa38009d90 tx=0x7faa3802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4a4d1230ee44ca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.394+0000 7faa3effd640 1 -- 172.21.15.52:0/542996696 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38007740 con 0x7faa400d35b0 2024-05-11T01:16:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.394+0000 7faa3effd640 1 -- 172.21.15.52:0/542996696 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa380078e0 con 0x7faa400d35b0 2024-05-11T01:16:19.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.394+0000 7faa3effd640 1 -- 172.21.15.52:0/542996696 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38007cb0 con 0x7faa400d35b0 2024-05-11T01:16:19.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.395+0000 7faa46054640 1 -- 172.21.15.52:0/542996696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa400d35b0 msgr2=0x7faa400b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.395+0000 7faa46054640 1 --2- 172.21.15.52:0/542996696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa400d35b0 0x7faa400b2600 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faa38009d90 tx=0x7faa3802a660 comp rx=0 tx=0).stop 2024-05-11T01:16:19.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.395+0000 7faa46054640 1 -- 172.21.15.52:0/542996696 shutdown_connections 2024-05-11T01:16:19.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.395+0000 7faa46054640 1 --2- 172.21.15.52:0/542996696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa400d35b0 0x7faa400b2600 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.395+0000 7faa46054640 1 --2- 172.21.15.52:0/542996696 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4015f2d0 0x7faa4006e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.395+0000 7faa46054640 1 -- 172.21.15.52:0/542996696 >> 172.21.15.52:0/542996696 conn(0x7faa4009ba20 msgr2=0x7faa4009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:19.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.395+0000 7faa46054640 1 -- 172.21.15.52:0/542996696 shutdown_connections 2024-05-11T01:16:19.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.395+0000 7faa46054640 1 -- 172.21.15.52:0/542996696 wait complete. 2024-05-11T01:16:19.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.396+0000 7faa46054640 1 Processor -- start 2024-05-11T01:16:19.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.396+0000 7faa46054640 1 -- start start 2024-05-11T01:16:19.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa46054640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4006e5d0 0x7faa4015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa46054640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa400d35b0 0x7faa400b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa46054640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa4015f2d0 0x7faa400b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa46054640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faa4015f830 con 0x7faa4015f2d0 2024-05-11T01:16:19.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa46054640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa4015f6b0 con 0x7faa400d35b0 2024-05-11T01:16:19.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa46054640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7faa400d1b80 con 0x7faa4006e5d0 2024-05-11T01:16:19.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa3ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa4015f2d0 0x7faa400b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa3ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa4015f2d0 0x7faa400b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43576/0 (socket says 172.21.15.52:43576) 2024-05-11T01:16:19.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa3ffff640 1 -- 172.21.15.52:0/1929055744 learned_addr learned my addr 172.21.15.52:0/1929055744 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:19.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa3ffff640 1 -- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4006e5d0 msgr2=0x7faa4015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa3ffff640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4006e5d0 0x7faa4015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa3ffff640 1 -- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa400d35b0 msgr2=0x7faa400b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa3ffff640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa400d35b0 0x7faa400b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.397+0000 7faa3ffff640 1 -- 172.21.15.52:0/1929055744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa400b7160 con 0x7faa4015f2d0 2024-05-11T01:16:19.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.398+0000 7faa3ffff640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa4015f2d0 0x7faa400b09f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faa38009eb0 tx=0x7faa38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.398+0000 7faa3cff9640 1 -- 172.21.15.52:0/1929055744 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38040070 con 0x7faa4015f2d0 2024-05-11T01:16:19.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.398+0000 7faa3cff9640 1 -- 172.21.15.52:0/1929055744 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa38031bc0 con 0x7faa4015f2d0 2024-05-11T01:16:19.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.398+0000 7faa3cff9640 1 -- 172.21.15.52:0/1929055744 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38009450 con 0x7faa4015f2d0 2024-05-11T01:16:19.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.398+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa400b7440 con 0x7faa4015f2d0 2024-05-11T01:16:19.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.398+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa400b7a00 con 0x7faa4015f2d0 2024-05-11T01:16:19.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.399+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa40069780 con 0x7faa4015f2d0 2024-05-11T01:16:19.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.399+0000 7faa3cff9640 1 -- 172.21.15.52:0/1929055744 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7faa38039040 con 0x7faa4015f2d0 2024-05-11T01:16:19.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.400+0000 7faa3cff9640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa200762d0 0x7faa20078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.400+0000 7faa3cff9640 1 -- 172.21.15.52:0/1929055744 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7faa380b9120 con 0x7faa4015f2d0 2024-05-11T01:16:19.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.400+0000 7faa3f7fe640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa200762d0 0x7faa20078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.401+0000 7faa3f7fe640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa200762d0 0x7faa20078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa30006a90 tx=0x7faa30006a20 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.404+0000 7faa3cff9640 1 -- 172.21.15.52:0/1929055744 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faa38082810 con 0x7faa4015f2d0 2024-05-11T01:16:19.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.597+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7faa400b3820 con 0x7faa200762d0 2024-05-11T01:16:19.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.598+0000 7faa3cff9640 1 -- 172.21.15.52:0/1929055744 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7faa400b3820 con 0x7faa200762d0 2024-05-11T01:16:19.598 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 4 to deep-scrub 2024-05-11T01:16:19.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa200762d0 msgr2=0x7faa20078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa200762d0 0x7faa20078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa30006a90 tx=0x7faa30006a20 comp rx=0 tx=0).stop 2024-05-11T01:16:19.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa4015f2d0 msgr2=0x7faa400b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa4015f2d0 0x7faa400b09f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faa38009eb0 tx=0x7faa38004a40 comp rx=0 tx=0).stop 2024-05-11T01:16:19.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 shutdown_connections 2024-05-11T01:16:19.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7faa200762d0 0x7faa20078790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa4015f2d0 0x7faa400b09f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa400d35b0 0x7faa400b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 --2- 172.21.15.52:0/1929055744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa4006e5d0 0x7faa4015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 >> 172.21.15.52:0/1929055744 conn(0x7faa4009ba20 msgr2=0x7faa4015e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:19.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 shutdown_connections 2024-05-11T01:16:19.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.600+0000 7faa46054640 1 -- 172.21.15.52:0/1929055744 wait complete. 2024-05-11T01:16:19.611 INFO:tasks.ceph:Scrubbing osd.5 2024-05-11T01:16:19.612 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-11T01:16:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.719+0000 7fa873415640 1 Processor -- start 2024-05-11T01:16:19.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.719+0000 7fa873415640 1 -- start start 2024-05-11T01:16:19.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa873415640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa86c0d57a0 0x7fa86c0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa873415640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa86c0b4dc0 0x7fa86c0d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa873415640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa86c0cfd40 con 0x7fa86c15f310 2024-05-11T01:16:19.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa873415640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7fa86c0b6140 con 0x7fa86c0b4dc0 2024-05-11T01:16:19.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa873415640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa86c0b5530 con 0x7fa86c0d57a0 2024-05-11T01:16:19.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa8719b2640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7fa86c0b4dc0 0x7fa86c0d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa8711b1640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7fa86c15f310 0x7fa86c0d5090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33786/0 (socket says 172.21.15.52:33786) 2024-05-11T01:16:19.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa8711b1640 1 -- 172.21.15.52:0/1975679737 learned_addr learned my addr 172.21.15.52:0/1975679737 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:19.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa8709b0640 1 --2- 172.21.15.52:0/1975679737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa86c0d57a0 0x7fa86c0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa8719b2640 1 -- 172.21.15.52:0/1975679737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa86c0d57a0 msgr2=0x7fa86c0d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:19.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa8719b2640 1 --2- 172.21.15.52:0/1975679737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa86c0d57a0 0x7fa86c0d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa8719b2640 1 -- 172.21.15.52:0/1975679737 >> v1:172.21.15.52:6789/0 conn(0x7fa86c15f310 legacy=0x7fa86c0d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.720+0000 7fa86bfff640 1 -- 172.21.15.52:0/1975679737 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 874649696 0 0) 0x7fa864001490 con 0x7fa86c15f310 2024-05-11T01:16:19.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.721+0000 7fa8719b2640 1 -- 172.21.15.52:0/1975679737 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c0b5810 con 0x7fa86c0b4dc0 2024-05-11T01:16:19.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.721+0000 7fa8719b2640 1 --2- 172.21.15.52:0/1975679737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c0b4dc0 0x7fa86c0d2f40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0097a0 tx=0x7fa85c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e72cbedeb9fd97 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.721+0000 7fa86bfff640 1 -- 172.21.15.52:0/1975679737 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c037070 con 0x7fa86c0b4dc0 2024-05-11T01:16:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.721+0000 7fa86bfff640 1 -- 172.21.15.52:0/1975679737 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa85c032440 con 0x7fa86c0b4dc0 2024-05-11T01:16:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.721+0000 7fa86bfff640 1 -- 172.21.15.52:0/1975679737 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c032810 con 0x7fa86c0b4dc0 2024-05-11T01:16:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.722+0000 7fa873415640 1 -- 172.21.15.52:0/1975679737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c0b4dc0 msgr2=0x7fa86c0d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.722+0000 7fa873415640 1 --2- 172.21.15.52:0/1975679737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c0b4dc0 0x7fa86c0d2f40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0097a0 tx=0x7fa85c02a180 comp rx=0 tx=0).stop 2024-05-11T01:16:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.722+0000 7fa873415640 1 -- 172.21.15.52:0/1975679737 shutdown_connections 2024-05-11T01:16:19.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.722+0000 7fa873415640 1 --2- 172.21.15.52:0/1975679737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c0b4dc0 0x7fa86c0d2f40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.722+0000 7fa873415640 1 --2- 172.21.15.52:0/1975679737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa86c0d57a0 0x7fa86c0d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.722+0000 7fa873415640 1 -- 172.21.15.52:0/1975679737 >> 172.21.15.52:0/1975679737 conn(0x7fa86c08eb80 msgr2=0x7fa86c09e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.722+0000 7fa873415640 1 -- 172.21.15.52:0/1975679737 shutdown_connections 2024-05-11T01:16:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.722+0000 7fa873415640 1 -- 172.21.15.52:0/1975679737 wait complete. 2024-05-11T01:16:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.723+0000 7fa873415640 1 Processor -- start 2024-05-11T01:16:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.723+0000 7fa873415640 1 -- start start 2024-05-11T01:16:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa873415640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa86c0b4dc0 0x7fa86c14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa873415640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa86c0d57a0 0x7fa86c0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa873415640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c15f310 0x7fa86c0d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa873415640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c0b5230 con 0x7fa86c15f310 2024-05-11T01:16:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa873415640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c0b5dd0 con 0x7fa86c0b4dc0 2024-05-11T01:16:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa873415640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa86c15f6f0 con 0x7fa86c0d57a0 2024-05-11T01:16:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa8719b2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c15f310 0x7fa86c0d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa8719b2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c15f310 0x7fa86c0d0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43594/0 (socket says 172.21.15.52:43594) 2024-05-11T01:16:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa8719b2640 1 -- 172.21.15.52:0/3691251374 learned_addr learned my addr 172.21.15.52:0/3691251374 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa8719b2640 1 -- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa86c0d57a0 msgr2=0x7fa86c0c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa8719b2640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa86c0d57a0 0x7fa86c0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa8719b2640 1 -- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa86c0b4dc0 msgr2=0x7fa86c14aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa8711b1640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa86c0b4dc0 0x7fa86c14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa8719b2640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa86c0b4dc0 0x7fa86c14aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.724+0000 7fa8719b2640 1 -- 172.21.15.52:0/3691251374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c1f8310 con 0x7fa86c15f310 2024-05-11T01:16:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.725+0000 7fa8719b2640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c15f310 0x7fa86c0d0c80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0098c0 tx=0x7fa85c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.725+0000 7fa869ffb640 1 -- 172.21.15.52:0/3691251374 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c037040 con 0x7fa86c15f310 2024-05-11T01:16:19.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.725+0000 7fa869ffb640 1 -- 172.21.15.52:0/3691251374 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa85c042070 con 0x7fa86c15f310 2024-05-11T01:16:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.725+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c14b000 con 0x7fa86c15f310 2024-05-11T01:16:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.725+0000 7fa869ffb640 1 -- 172.21.15.52:0/3691251374 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c030b50 con 0x7fa86c15f310 2024-05-11T01:16:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.725+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c1f88b0 con 0x7fa86c15f310 2024-05-11T01:16:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.726+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa86c069780 con 0x7fa86c15f310 2024-05-11T01:16:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.727+0000 7fa869ffb640 1 -- 172.21.15.52:0/3691251374 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fa85c030cf0 con 0x7fa86c15f310 2024-05-11T01:16:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.727+0000 7fa869ffb640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa8580762d0 0x7fa858078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.727+0000 7fa869ffb640 1 -- 172.21.15.52:0/3691251374 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7fa85c0b8d40 con 0x7fa86c15f310 2024-05-11T01:16:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.727+0000 7fa869ffb640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fa85807ea00 0x7fa858080e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.727+0000 7fa869ffb640 1 -- 172.21.15.52:0/3691251374 --> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa85c032440 con 0x7fa85807ea00 2024-05-11T01:16:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.727+0000 7fa869ffb640 1 -- 172.21.15.52:0/3691251374 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fa85c0b91d0 con 0x7fa86c15f310 2024-05-11T01:16:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.727+0000 7fa8711b1640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa8580762d0 0x7fa858078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.728+0000 7fa8709b0640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fa85807ea00 0x7fa858080e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.728+0000 7fa8711b1640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa8580762d0 0x7fa858078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8640012e0 tx=0x7fa864009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.728+0000 7fa8709b0640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fa85807ea00 0x7fa858080e10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.729+0000 7fa869ffb640 1 -- 172.21.15.52:0/3691251374 <== osd.5 v2:172.21.15.119:6824/3499512602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa85c032440 con 0x7fa85807ea00 2024-05-11T01:16:19.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.742+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 --> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa86c068830 con 0x7fa85807ea00 2024-05-11T01:16:19.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa869ffb640 1 -- 172.21.15.52:0/3691251374 <== osd.5 v2:172.21.15.119:6824/3499512602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa86c068830 con 0x7fa85807ea00 2024-05-11T01:16:19.743 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:16:19.743 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T01:16:19.743 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:16:19.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fa85807ea00 msgr2=0x7fa858080e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6824/3499512602,v1:172.21.15.119:6825/3499512602] conn(0x7fa85807ea00 0x7fa858080e10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa8580762d0 msgr2=0x7fa858078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa8580762d0 0x7fa858078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8640012e0 tx=0x7fa864009040 comp rx=0 tx=0).stop 2024-05-11T01:16:19.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c15f310 msgr2=0x7fa86c0d0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 --2- 172.21.15.52:0/3691251374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa86c15f310 0x7fa86c0d0c80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0098c0 tx=0x7fa85c0046c0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa8719b2640 1 -- 172.21.15.52:0/3691251374 reap_dead start 2024-05-11T01:16:19.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 shutdown_connections 2024-05-11T01:16:19.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 >> 172.21.15.52:0/3691251374 conn(0x7fa86c08eb80 msgr2=0x7fa86c15da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:19.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 shutdown_connections 2024-05-11T01:16:19.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.744+0000 7fa873415640 1 -- 172.21.15.52:0/3691251374 wait complete. 2024-05-11T01:16:19.754 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-11T01:16:19.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.859+0000 7f70805c8640 1 Processor -- start 2024-05-11T01:16:19.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.859+0000 7f70805c8640 1 -- start start 2024-05-11T01:16:19.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.860+0000 7f70805c8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70780d3330 0x7f70780b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.860+0000 7f70805c8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70780d1f90 con 0x7f70780d2ea0 2024-05-11T01:16:19.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.860+0000 7f70805c8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70780d2270 con 0x7f707815f2d0 2024-05-11T01:16:19.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.860+0000 7f70805c8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f70780d2550 con 0x7f70780d3330 2024-05-11T01:16:19.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.860+0000 7f707db63640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f70780d2ea0 0x7f70780b2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33794/0 (socket says 172.21.15.52:33794) 2024-05-11T01:16:19.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.860+0000 7f707db63640 1 -- 172.21.15.52:0/616145993 learned_addr learned my addr 172.21.15.52:0/616145993 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:19.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.860+0000 7f707eb65640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70780d3330 0x7f70780b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.860+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 <== mon.0 v1:172.21.15.52:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3616334944 0 0) 0x7f70780d1f90 con 0x7f70780d2ea0 2024-05-11T01:16:19.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.860+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 --> v1:172.21.15.52:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7064003520 con 0x7f70780d2ea0 2024-05-11T01:16:19.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215878294 0 0) 0x7f70780d2270 con 0x7f707815f2d0 2024-05-11T01:16:19.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70780d1f90 con 0x7f707815f2d0 2024-05-11T01:16:19.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 <== mon.0 v1:172.21.15.52:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1456174087 0 0) 0x7f7064003520 con 0x7f70780d2ea0 2024-05-11T01:16:19.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70780d3330 msgr2=0x7f70780b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:19.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70780d3330 0x7f70780b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 >> v1:172.21.15.119:6789/0 conn(0x7f707815f2d0 legacy=0x7f707806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 --> v1:172.21.15.52:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70780d2830 con 0x7f70780d2ea0 2024-05-11T01:16:19.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 <== mon.0 v1:172.21.15.52:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1933429842 0 0) 0x7f7068001670 con 0x7f70780d2ea0 2024-05-11T01:16:19.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 >> v1:172.21.15.52:6789/0 conn(0x7f70780d2ea0 legacy=0x7f70780b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70780d3330 0x7f7064003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004210 0x7f7064004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7064004d90 0x7f7064005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70780d2550 con 0x7f7064004d90 2024-05-11T01:16:19.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f706400bbd0 con 0x7f7064004210 2024-05-11T01:16:19.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f706400bdd0 con 0x7f70780d3330 2024-05-11T01:16:19.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707eb65640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7064004d90 0x7f7064005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707db63640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004210 0x7f7064004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707e364640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70780d3330 0x7f7064003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707eb65640 1 -- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70780d3330 msgr2=0x7f7064003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:19.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707eb65640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70780d3330 0x7f7064003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707eb65640 1 -- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004210 msgr2=0x7f7064004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:19.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707eb65640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004210 0x7f7064004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.861+0000 7f707eb65640 1 -- 172.21.15.52:0/616145993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7064003520 con 0x7f7064004d90 2024-05-11T01:16:19.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.862+0000 7f707eb65640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7064004d90 0x7f7064005210 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7070009fc0 tx=0x7f7064005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c95129d2bc0431 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.862+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070009c10 con 0x7f7064004d90 2024-05-11T01:16:19.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.862+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7070036070 con 0x7f7064004d90 2024-05-11T01:16:19.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.862+0000 7f707d362640 1 -- 172.21.15.52:0/616145993 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070031880 con 0x7f7064004d90 2024-05-11T01:16:19.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.863+0000 7f70805c8640 1 -- 172.21.15.52:0/616145993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7064004d90 msgr2=0x7f7064005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.863+0000 7f70805c8640 1 --2- 172.21.15.52:0/616145993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7064004d90 0x7f7064005210 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7070009fc0 tx=0x7f7064005830 comp rx=0 tx=0).stop 2024-05-11T01:16:19.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.863+0000 7f707eb65640 1 -- 172.21.15.52:0/616145993 reap_dead start 2024-05-11T01:16:19.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.863+0000 7f70805c8640 1 -- 172.21.15.52:0/616145993 shutdown_connections 2024-05-11T01:16:19.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.863+0000 7f70805c8640 1 -- 172.21.15.52:0/616145993 >> 172.21.15.52:0/616145993 conn(0x7f707809ba20 msgr2=0x7f707809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:19.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.863+0000 7f70805c8640 1 -- 172.21.15.52:0/616145993 shutdown_connections 2024-05-11T01:16:19.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.863+0000 7f70805c8640 1 -- 172.21.15.52:0/616145993 wait complete. 2024-05-11T01:16:19.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.864+0000 7f70805c8640 1 Processor -- start 2024-05-11T01:16:19.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.864+0000 7f70805c8640 1 -- start start 2024-05-11T01:16:19.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f70805c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004d90 0x7f70781f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f70805c8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7078144d30 0x7f7078157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f70805c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7078162040 0x7f70781624a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f70805c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f707815f6b0 con 0x7f7078162040 2024-05-11T01:16:19.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f70805c8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f70780d3740 con 0x7f7064004d90 2024-05-11T01:16:19.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f70805c8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f707815f830 con 0x7f7078144d30 2024-05-11T01:16:19.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707eb65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7078162040 0x7f70781624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707eb65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7078162040 0x7f70781624a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43602/0 (socket says 172.21.15.52:43602) 2024-05-11T01:16:19.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707eb65640 1 -- 172.21.15.52:0/2155838372 learned_addr learned my addr 172.21.15.52:0/2155838372 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:19.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707db63640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7078144d30 0x7f7078157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707e364640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004d90 0x7f70781f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707eb65640 1 -- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7078144d30 msgr2=0x7f7078157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707eb65640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7078144d30 0x7f7078157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707eb65640 1 -- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004d90 msgr2=0x7f70781f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:19.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707eb65640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004d90 0x7f70781f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:19.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707eb65640 1 -- 172.21.15.52:0/2155838372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70781451b0 con 0x7f7078162040 2024-05-11T01:16:19.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707db63640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7078144d30 0x7f7078157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:19.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707e364640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004d90 0x7f70781f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:19.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.865+0000 7f707eb65640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7078162040 0x7f70781624a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7070033040 tx=0x7f7070009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.866+0000 7f706effd640 1 -- 172.21.15.52:0/2155838372 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070009c10 con 0x7f7078162040 2024-05-11T01:16:19.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.866+0000 7f706effd640 1 -- 172.21.15.52:0/2155838372 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7070031db0 con 0x7f7078162040 2024-05-11T01:16:19.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.866+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7078157f80 con 0x7f7078162040 2024-05-11T01:16:19.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.866+0000 7f706effd640 1 -- 172.21.15.52:0/2155838372 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070003bd0 con 0x7f7078162040 2024-05-11T01:16:19.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.866+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70780b7090 con 0x7f7078162040 2024-05-11T01:16:19.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.867+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70780b74d0 con 0x7f7078162040 2024-05-11T01:16:19.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.867+0000 7f706effd640 1 -- 172.21.15.52:0/2155838372 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f7070005d10 con 0x7f7078162040 2024-05-11T01:16:19.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.868+0000 7f706effd640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f705c076300 0x7f705c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:19.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.868+0000 7f706effd640 1 -- 172.21.15.52:0/2155838372 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f70700b8f20 con 0x7f7078162040 2024-05-11T01:16:19.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.868+0000 7f707e364640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f705c076300 0x7f705c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:19.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.868+0000 7f707e364640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f705c076300 0x7f705c0787c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f70740036b0 tx=0x7f7074009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:19.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:19.872+0000 7f706effd640 1 -- 172.21.15.52:0/2155838372 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f707004a060 con 0x7f7078162040 2024-05-11T01:16:20.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.068+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f70780b6ba0 con 0x7f705c076300 2024-05-11T01:16:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.070+0000 7f706effd640 1 -- 172.21.15.52:0/2155838372 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f70780b6ba0 con 0x7f705c076300 2024-05-11T01:16:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 5 to deep-scrub 2024-05-11T01:16:20.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f705c076300 msgr2=0x7f705c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f705c076300 0x7f705c0787c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f70740036b0 tx=0x7f7074009040 comp rx=0 tx=0).stop 2024-05-11T01:16:20.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7078162040 msgr2=0x7f70781624a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7078162040 0x7f70781624a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7070033040 tx=0x7f7070009750 comp rx=0 tx=0).stop 2024-05-11T01:16:20.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 shutdown_connections 2024-05-11T01:16:20.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f705c076300 0x7f705c0787c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7078162040 0x7f70781624a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7078144d30 0x7f7078157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 --2- 172.21.15.52:0/2155838372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7064004d90 0x7f70781f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 >> 172.21.15.52:0/2155838372 conn(0x7f707809ba20 msgr2=0x7f707806d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:20.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 shutdown_connections 2024-05-11T01:16:20.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.071+0000 7f70805c8640 1 -- 172.21.15.52:0/2155838372 wait complete. 2024-05-11T01:16:20.082 INFO:tasks.ceph:Scrubbing osd.6 2024-05-11T01:16:20.083 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-11T01:16:20.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.190+0000 7f8b81946640 1 Processor -- start 2024-05-11T01:16:20.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.190+0000 7f8b81946640 1 -- start start 2024-05-11T01:16:20.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b81946640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8b7c09ba20 0x7f8b7c09be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b81946640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8b7c0b11d0 0x7f8b7c0b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b81946640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b7c0afe50 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b81946640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8b7c09c3c0 con 0x7f8b7c0b11d0 2024-05-11T01:16:20.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b81946640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8b7c0b0d40 con 0x7f8b7c09ba20 2024-05-11T01:16:20.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b7b7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8b7c0b11d0 0x7f8b7c0b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b7b7fe640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8b7c0b11d0 0x7f8b7c0b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43604/0 (socket says 172.21.15.52:43604) 2024-05-11T01:16:20.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b7b7fe640 1 -- 172.21.15.52:0/3385685679 learned_addr learned my addr 172.21.15.52:0/3385685679 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:20.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b73fff640 1 --2- 172.21.15.52:0/3385685679 >> v2:172.21.15.119:3300/0 conn(0x7f8b7c09ba20 0x7f8b7c09be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b7b7fe640 1 -- 172.21.15.52:0/3385685679 >> v2:172.21.15.119:3300/0 conn(0x7f8b7c09ba20 msgr2=0x7f8b7c09be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:20.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b7b7fe640 1 --2- 172.21.15.52:0/3385685679 >> v2:172.21.15.119:3300/0 conn(0x7f8b7c09ba20 0x7f8b7c09be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b7b7fe640 1 -- 172.21.15.52:0/3385685679 >> v1:172.21.15.119:6789/0 conn(0x7f8b7c15d2e0 legacy=0x7f8b7c071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.191+0000 7f8b7b7fe640 1 -- 172.21.15.52:0/3385685679 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c0b0130 con 0x7f8b7c0b11d0 2024-05-11T01:16:20.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.192+0000 7f8b7b7fe640 1 --2- 172.21.15.52:0/3385685679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c0b11d0 0x7f8b7c0b1660 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c072f10 tx=0x7f8b7402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e9cb1f1087510d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.192+0000 7f8b7a7fc640 1 -- 172.21.15.52:0/3385685679 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74036070 con 0x7f8b7c0b11d0 2024-05-11T01:16:20.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.192+0000 7f8b7a7fc640 1 -- 172.21.15.52:0/3385685679 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b74031600 con 0x7f8b7c0b11d0 2024-05-11T01:16:20.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.192+0000 7f8b7a7fc640 1 -- 172.21.15.52:0/3385685679 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b740319d0 con 0x7f8b7c0b11d0 2024-05-11T01:16:20.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.193+0000 7f8b81946640 1 -- 172.21.15.52:0/3385685679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c0b11d0 msgr2=0x7f8b7c0b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.193+0000 7f8b81946640 1 --2- 172.21.15.52:0/3385685679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c0b11d0 0x7f8b7c0b1660 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c072f10 tx=0x7f8b7402a730 comp rx=0 tx=0).stop 2024-05-11T01:16:20.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.193+0000 7f8b81946640 1 -- 172.21.15.52:0/3385685679 shutdown_connections 2024-05-11T01:16:20.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.193+0000 7f8b81946640 1 --2- 172.21.15.52:0/3385685679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c0b11d0 0x7f8b7c0b1660 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.193+0000 7f8b81946640 1 --2- 172.21.15.52:0/3385685679 >> v2:172.21.15.119:3300/0 conn(0x7f8b7c09ba20 0x7f8b7c09be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.193+0000 7f8b81946640 1 -- 172.21.15.52:0/3385685679 >> 172.21.15.52:0/3385685679 conn(0x7f8b7c0d3160 msgr2=0x7f8b7c0d3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:20.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.193+0000 7f8b81946640 1 -- 172.21.15.52:0/3385685679 shutdown_connections 2024-05-11T01:16:20.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.193+0000 7f8b81946640 1 -- 172.21.15.52:0/3385685679 wait complete. 2024-05-11T01:16:20.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.194+0000 7f8b81946640 1 Processor -- start 2024-05-11T01:16:20.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.194+0000 7f8b81946640 1 -- start start 2024-05-11T01:16:20.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b81946640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8b7c09ba20 0x7f8b7c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b81946640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b7c0b11d0 0x7f8b7c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b81946640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c15d2e0 0x7f8b7c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b81946640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c0b09d0 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b81946640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c0b06f0 con 0x7f8b7c0b11d0 2024-05-11T01:16:20.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b81946640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c072160 con 0x7f8b7c09ba20 2024-05-11T01:16:20.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c15d2e0 0x7f8b7c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c15d2e0 0x7f8b7c0b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43614/0 (socket says 172.21.15.52:43614) 2024-05-11T01:16:20.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7b7fe640 1 -- 172.21.15.52:0/2280895116 learned_addr learned my addr 172.21.15.52:0/2280895116 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:20.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b73fff640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b7c0b11d0 0x7f8b7c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7affd640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8b7c09ba20 0x7f8b7c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7b7fe640 1 -- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8b7c09ba20 msgr2=0x7f8b7c15f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7b7fe640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8b7c09ba20 0x7f8b7c15f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7b7fe640 1 -- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b7c0b11d0 msgr2=0x7f8b7c0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7b7fe640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b7c0b11d0 0x7f8b7c0b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7b7fe640 1 -- 172.21.15.52:0/2280895116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c160330 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.195+0000 7f8b7b7fe640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c15d2e0 0x7f8b7c0b4180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009f40 tx=0x7f8b74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.196+0000 7f8b78ff9640 1 -- 172.21.15.52:0/2280895116 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74036050 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.196+0000 7f8b78ff9640 1 -- 172.21.15.52:0/2280895116 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b74042070 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.196+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7c1f33f0 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.196+0000 7f8b78ff9640 1 -- 172.21.15.52:0/2280895116 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74009510 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.196+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7c158c20 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.197+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b7c1f3ab0 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.197+0000 7f8b78ff9640 1 -- 172.21.15.52:0/2280895116 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f8b74007bf0 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.198+0000 7f8b78ff9640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8b5c076300 0x7f8b5c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.198+0000 7f8b78ff9640 1 -- 172.21.15.52:0/2280895116 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f8b740b85a0 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.198+0000 7f8b78ff9640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7f8b5c07ea30 0x7f8b5c080e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.198+0000 7f8b78ff9640 1 -- 172.21.15.52:0/2280895116 --> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b5c081470 con 0x7f8b5c07ea30 2024-05-11T01:16:20.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.198+0000 7f8b78ff9640 1 -- 172.21.15.52:0/2280895116 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8b740bf050 con 0x7f8b7c15d2e0 2024-05-11T01:16:20.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.198+0000 7f8b7affd640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8b5c076300 0x7f8b5c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.198+0000 7f8b73fff640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7f8b5c07ea30 0x7f8b5c080e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.199+0000 7f8b7affd640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8b5c076300 0x7f8b5c0787c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c000f60 tx=0x7f8b6c009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.199+0000 7f8b73fff640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7f8b5c07ea30 0x7f8b5c080e40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.200+0000 7f8b78ff9640 1 -- 172.21.15.52:0/2280895116 <== osd.6 v2:172.21.15.119:6808/2126488559 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b5c081470 con 0x7f8b5c07ea30 2024-05-11T01:16:20.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.213+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 --> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8b7c069780 con 0x7f8b5c07ea30 2024-05-11T01:16:20.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.213+0000 7f8b78ff9640 1 -- 172.21.15.52:0/2280895116 <== osd.6 v2:172.21.15.119:6808/2126488559 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8b7c069780 con 0x7f8b5c07ea30 2024-05-11T01:16:20.213 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:16:20.214 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T01:16:20.214 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:16:20.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.213+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7f8b5c07ea30 msgr2=0x7f8b5c080e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.213+0000 7f8b81946640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6808/2126488559,v1:172.21.15.119:6809/2126488559] conn(0x7f8b5c07ea30 0x7f8b5c080e40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.214+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8b5c076300 msgr2=0x7f8b5c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.214+0000 7f8b81946640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f8b5c076300 0x7f8b5c0787c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c000f60 tx=0x7f8b6c009040 comp rx=0 tx=0).stop 2024-05-11T01:16:20.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.214+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c15d2e0 msgr2=0x7f8b7c0b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.214+0000 7f8b81946640 1 --2- 172.21.15.52:0/2280895116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b7c15d2e0 0x7f8b7c0b4180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009f40 tx=0x7f8b74004a40 comp rx=0 tx=0).stop 2024-05-11T01:16:20.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.214+0000 7f8b7b7fe640 1 -- 172.21.15.52:0/2280895116 reap_dead start 2024-05-11T01:16:20.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.214+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 shutdown_connections 2024-05-11T01:16:20.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.214+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 >> 172.21.15.52:0/2280895116 conn(0x7f8b7c0d3160 msgr2=0x7f8b7c159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:20.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.214+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 shutdown_connections 2024-05-11T01:16:20.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.214+0000 7f8b81946640 1 -- 172.21.15.52:0/2280895116 wait complete. 2024-05-11T01:16:20.224 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-11T01:16:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.330+0000 7ff7efba5640 1 Processor -- start 2024-05-11T01:16:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.330+0000 7ff7efba5640 1 -- start start 2024-05-11T01:16:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7efba5640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff7e8160480 0x7ff7e80d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7efba5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e80d32c0 0x7ff7e80d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7efba5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7e8163250 con 0x7ff7e815f430 2024-05-11T01:16:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7efba5640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff7e8163530 con 0x7ff7e8160480 2024-05-11T01:16:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7efba5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e8163810 con 0x7ff7e80d32c0 2024-05-11T01:16:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7ed140640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff7e8160480 0x7ff7e80d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7ed140640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff7e8160480 0x7ff7e80d2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43622/0 (socket says 172.21.15.52:43622) 2024-05-11T01:16:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7ed140640 1 -- 172.21.15.52:0/2586803536 learned_addr learned my addr 172.21.15.52:0/2586803536 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7ed140640 1 -- 172.21.15.52:0/2586803536 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e80d32c0 msgr2=0x7ff7e80d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7ee142640 1 --2- 172.21.15.52:0/2586803536 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e80d32c0 0x7ff7e80d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7ed140640 1 --2- 172.21.15.52:0/2586803536 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e80d32c0 0x7ff7e80d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7ed140640 1 -- 172.21.15.52:0/2586803536 >> v1:172.21.15.119:6789/0 conn(0x7ff7e815f430 legacy=0x7ff7e815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7ed140640 1 -- 172.21.15.52:0/2586803536 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e8163af0 con 0x7ff7e8160480 2024-05-11T01:16:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.331+0000 7ff7ee142640 1 --2- 172.21.15.52:0/2586803536 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e80d32c0 0x7ff7e80d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.332+0000 7ff7ed140640 1 --2- 172.21.15.52:0/2586803536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 0x7ff7e80d2d80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80b4840 tx=0x7ff7d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b651466a4cf2e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.332+0000 7ff7ec93f640 1 -- 172.21.15.52:0/2586803536 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8037070 con 0x7ff7e8160480 2024-05-11T01:16:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.332+0000 7ff7ec93f640 1 -- 172.21.15.52:0/2586803536 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d8032440 con 0x7ff7e8160480 2024-05-11T01:16:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.332+0000 7ff7ec93f640 1 -- 172.21.15.52:0/2586803536 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8032810 con 0x7ff7e8160480 2024-05-11T01:16:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.333+0000 7ff7efba5640 1 -- 172.21.15.52:0/2586803536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 msgr2=0x7ff7e80d2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.333+0000 7ff7efba5640 1 --2- 172.21.15.52:0/2586803536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 0x7ff7e80d2d80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80b4840 tx=0x7ff7d802a180 comp rx=0 tx=0).stop 2024-05-11T01:16:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.333+0000 7ff7efba5640 1 -- 172.21.15.52:0/2586803536 shutdown_connections 2024-05-11T01:16:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.333+0000 7ff7efba5640 1 --2- 172.21.15.52:0/2586803536 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e80d32c0 0x7ff7e80d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.333+0000 7ff7efba5640 1 --2- 172.21.15.52:0/2586803536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 0x7ff7e80d2d80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.333+0000 7ff7efba5640 1 -- 172.21.15.52:0/2586803536 >> 172.21.15.52:0/2586803536 conn(0x7ff7e808eb70 msgr2=0x7ff7e809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.333+0000 7ff7efba5640 1 -- 172.21.15.52:0/2586803536 shutdown_connections 2024-05-11T01:16:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.333+0000 7ff7efba5640 1 -- 172.21.15.52:0/2586803536 wait complete. 2024-05-11T01:16:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.334+0000 7ff7efba5640 1 Processor -- start 2024-05-11T01:16:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.334+0000 7ff7efba5640 1 -- start start 2024-05-11T01:16:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7efba5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80d32c0 0x7ff7e815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7efba5640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e815f430 0x7ff7e80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7efba5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 0x7ff7e8161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7efba5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8163030 con 0x7ff7e8160480 2024-05-11T01:16:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7efba5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8162eb0 con 0x7ff7e80d32c0 2024-05-11T01:16:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7efba5640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e81608f0 con 0x7ff7e815f430 2024-05-11T01:16:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ee142640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 0x7ff7e8161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ee142640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 0x7ff7e8161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43630/0 (socket says 172.21.15.52:43630) 2024-05-11T01:16:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ee142640 1 -- 172.21.15.52:0/872194912 learned_addr learned my addr 172.21.15.52:0/872194912 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ed941640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80d32c0 0x7ff7e815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ed140640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e815f430 0x7ff7e80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ee142640 1 -- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e815f430 msgr2=0x7ff7e80cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ee142640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e815f430 0x7ff7e80cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ee142640 1 -- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80d32c0 msgr2=0x7ff7e815a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ee142640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80d32c0 0x7ff7e815a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ee142640 1 -- 172.21.15.52:0/872194912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e80cc750 con 0x7ff7e8160480 2024-05-11T01:16:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ed941640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80d32c0 0x7ff7e815a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.335+0000 7ff7ee142640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 0x7ff7e8161e00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000cf20 tx=0x7ff7e000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.336+0000 7ff7de7fc640 1 -- 172.21.15.52:0/872194912 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e001a070 con 0x7ff7e8160480 2024-05-11T01:16:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.336+0000 7ff7de7fc640 1 -- 172.21.15.52:0/872194912 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e000d840 con 0x7ff7e8160480 2024-05-11T01:16:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.336+0000 7ff7de7fc640 1 -- 172.21.15.52:0/872194912 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e000dc30 con 0x7ff7e8160480 2024-05-11T01:16:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.336+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e80cc9d0 con 0x7ff7e8160480 2024-05-11T01:16:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.336+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e80ccf90 con 0x7ff7e8160480 2024-05-11T01:16:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.337+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e8069780 con 0x7ff7e8160480 2024-05-11T01:16:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.337+0000 7ff7de7fc640 1 -- 172.21.15.52:0/872194912 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7ff7e001d070 con 0x7ff7e8160480 2024-05-11T01:16:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.338+0000 7ff7de7fc640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff7cc076300 0x7ff7cc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.338+0000 7ff7de7fc640 1 -- 172.21.15.52:0/872194912 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7ff7e0098190 con 0x7ff7e8160480 2024-05-11T01:16:20.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.338+0000 7ff7ed941640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff7cc076300 0x7ff7cc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.338+0000 7ff7ed941640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff7cc076300 0x7ff7cc0787c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4003eb0 tx=0x7ff7e4009040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.342+0000 7ff7de7fc640 1 -- 172.21.15.52:0/872194912 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff7e80cc750 con 0x7ff7e8160480 2024-05-11T01:16:20.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.535+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7ff7e80b4ba0 con 0x7ff7cc076300 2024-05-11T01:16:20.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.535+0000 7ff7de7fc640 1 -- 172.21.15.52:0/872194912 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff7e80b4ba0 con 0x7ff7cc076300 2024-05-11T01:16:20.536 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 6 to deep-scrub 2024-05-11T01:16:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.537+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff7cc076300 msgr2=0x7ff7cc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.537+0000 7ff7efba5640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff7cc076300 0x7ff7cc0787c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4003eb0 tx=0x7ff7e4009040 comp rx=0 tx=0).stop 2024-05-11T01:16:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.537+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 msgr2=0x7ff7e8161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.537+0000 7ff7efba5640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 0x7ff7e8161e00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000cf20 tx=0x7ff7e000a7d0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.538+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 shutdown_connections 2024-05-11T01:16:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.538+0000 7ff7efba5640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff7cc076300 0x7ff7cc0787c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.538+0000 7ff7efba5640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7e8160480 0x7ff7e8161e00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.538+0000 7ff7efba5640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7e815f430 0x7ff7e80cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.538+0000 7ff7efba5640 1 --2- 172.21.15.52:0/872194912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80d32c0 0x7ff7e815a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.538+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 >> 172.21.15.52:0/872194912 conn(0x7ff7e808eb70 msgr2=0x7ff7e809e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.538+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 shutdown_connections 2024-05-11T01:16:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.538+0000 7ff7efba5640 1 -- 172.21.15.52:0/872194912 wait complete. 2024-05-11T01:16:20.549 INFO:tasks.ceph:Scrubbing osd.7 2024-05-11T01:16:20.549 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-11T01:16:20.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.656+0000 7f879d7c8640 1 Processor -- start 2024-05-11T01:16:20.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.657+0000 7f879d7c8640 1 -- start start 2024-05-11T01:16:20.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.657+0000 7f879d7c8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f879815d0b0 0x7f8798072120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.657+0000 7f879d7c8640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8798073f00 0x7f87980cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.657+0000 7f879d7c8640 1 -- --> v1:172.21.15.52:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87980cbce0 con 0x7f87980cb8c0 2024-05-11T01:16:20.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.657+0000 7f879d7c8640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f8798160d50 con 0x7f8798073f00 2024-05-11T01:16:20.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.657+0000 7f879d7c8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7f8798161030 con 0x7f879815d0b0 2024-05-11T01:16:20.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.657+0000 7f87967fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8798073f00 0x7f87980cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.658+0000 7f87977fe640 1 --1- >> v1:172.21.15.52:6789/0 conn(0x7f87980cb8c0 0x7f879815d490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.52:6789/0 says I am v1:172.21.15.52:33808/0 (socket says 172.21.15.52:33808) 2024-05-11T01:16:20.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.658+0000 7f87977fe640 1 -- 172.21.15.52:0/1444666519 learned_addr learned my addr 172.21.15.52:0/1444666519 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:20.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.658+0000 7f8796ffd640 1 --2- 172.21.15.52:0/1444666519 >> v2:172.21.15.119:3300/0 conn(0x7f879815d0b0 0x7f8798072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.658+0000 7f87967fc640 1 -- 172.21.15.52:0/1444666519 >> v2:172.21.15.119:3300/0 conn(0x7f879815d0b0 msgr2=0x7f8798072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:20.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.658+0000 7f87967fc640 1 --2- 172.21.15.52:0/1444666519 >> v2:172.21.15.119:3300/0 conn(0x7f879815d0b0 0x7f8798072120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.658+0000 7f87967fc640 1 -- 172.21.15.52:0/1444666519 >> v1:172.21.15.52:6789/0 conn(0x7f87980cb8c0 legacy=0x7f879815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.658+0000 7f87967fc640 1 -- 172.21.15.52:0/1444666519 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8798161310 con 0x7f8798073f00 2024-05-11T01:16:20.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.658+0000 7f8796ffd640 1 --2- 172.21.15.52:0/1444666519 >> v2:172.21.15.119:3300/0 conn(0x7f879815d0b0 0x7f8798072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:20.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f87967fc640 1 --2- 172.21.15.52:0/1444666519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 0x7f87980cb380 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f879806d5b0 tx=0x7f878c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76cdb6ab9c2fddd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f8795ffb640 1 -- 172.21.15.52:0/1444666519 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c037070 con 0x7f8798073f00 2024-05-11T01:16:20.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f8795ffb640 1 -- 172.21.15.52:0/1444666519 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878c032440 con 0x7f8798073f00 2024-05-11T01:16:20.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f8795ffb640 1 -- 172.21.15.52:0/1444666519 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c032810 con 0x7f8798073f00 2024-05-11T01:16:20.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f879d7c8640 1 -- 172.21.15.52:0/1444666519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 msgr2=0x7f87980cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f879d7c8640 1 --2- 172.21.15.52:0/1444666519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 0x7f87980cb380 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f879806d5b0 tx=0x7f878c02a1a0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f879d7c8640 1 -- 172.21.15.52:0/1444666519 shutdown_connections 2024-05-11T01:16:20.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f879d7c8640 1 --2- 172.21.15.52:0/1444666519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 0x7f87980cb380 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f879d7c8640 1 --2- 172.21.15.52:0/1444666519 >> v2:172.21.15.119:3300/0 conn(0x7f879815d0b0 0x7f8798072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f879d7c8640 1 -- 172.21.15.52:0/1444666519 >> 172.21.15.52:0/1444666519 conn(0x7f87980cd7b0 msgr2=0x7f87980cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:20.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f879d7c8640 1 -- 172.21.15.52:0/1444666519 shutdown_connections 2024-05-11T01:16:20.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.659+0000 7f879d7c8640 1 -- 172.21.15.52:0/1444666519 wait complete. 2024-05-11T01:16:20.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.661+0000 7f879d7c8640 1 Processor -- start 2024-05-11T01:16:20.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.661+0000 7f879d7c8640 1 -- start start 2024-05-11T01:16:20.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.661+0000 7f879d7c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 0x7f8798152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.661+0000 7f879d7c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87980cb8c0 0x7f87980c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.661+0000 7f879d7c8640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f879815d0b0 0x7f87980c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.661+0000 7f879d7c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8798072660 con 0x7f8798073f00 2024-05-11T01:16:20.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.661+0000 7f879d7c8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f87981618d0 con 0x7f87980cb8c0 2024-05-11T01:16:20.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.661+0000 7f879d7c8640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f87980727e0 con 0x7f879815d0b0 2024-05-11T01:16:20.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f8796ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 0x7f8798152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f8796ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 0x7f8798152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43648/0 (socket says 172.21.15.52:43648) 2024-05-11T01:16:20.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f8796ffd640 1 -- 172.21.15.52:0/1527254646 learned_addr learned my addr 172.21.15.52:0/1527254646 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:20.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f87967fc640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87980cb8c0 0x7f87980c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f87977fe640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f879815d0b0 0x7f87980c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f8796ffd640 1 -- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f879815d0b0 msgr2=0x7f87980c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f8796ffd640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f879815d0b0 0x7f87980c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f8796ffd640 1 -- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87980cb8c0 msgr2=0x7f87980c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f8796ffd640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87980cb8c0 0x7f87980c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f8796ffd640 1 -- 172.21.15.52:0/1527254646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87980ce9a0 con 0x7f8798073f00 2024-05-11T01:16:20.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f87967fc640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87980cb8c0 0x7f87980c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:20.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f8796ffd640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 0x7f8798152500 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f878800ce30 tx=0x7f878800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f877f7fe640 1 -- 172.21.15.52:0/1527254646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878800f890 con 0x7f8798073f00 2024-05-11T01:16:20.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f877f7fe640 1 -- 172.21.15.52:0/1527254646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878800fa30 con 0x7f8798073f00 2024-05-11T01:16:20.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87980d02e0 con 0x7f8798073f00 2024-05-11T01:16:20.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f877f7fe640 1 -- 172.21.15.52:0/1527254646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878800fe00 con 0x7f8798073f00 2024-05-11T01:16:20.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.662+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87980d0860 con 0x7f8798073f00 2024-05-11T01:16:20.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.663+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8798152a40 con 0x7f8798073f00 2024-05-11T01:16:20.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.664+0000 7f877f7fe640 1 -- 172.21.15.52:0/1527254646 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f8788018400 con 0x7f8798073f00 2024-05-11T01:16:20.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.664+0000 7f877f7fe640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87840762d0 0x7f8784078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.664+0000 7f877f7fe640 1 -- 172.21.15.52:0/1527254646 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f8788098580 con 0x7f8798073f00 2024-05-11T01:16:20.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.664+0000 7f877f7fe640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f878407ea00 0x7f8784080e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.664+0000 7f87967fc640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87840762d0 0x7f8784078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.665+0000 7f877f7fe640 1 -- 172.21.15.52:0/1527254646 --> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8788011e50 con 0x7f878407ea00 2024-05-11T01:16:20.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.665+0000 7f877f7fe640 1 -- 172.21.15.52:0/1527254646 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f878809f050 con 0x7f8798073f00 2024-05-11T01:16:20.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.665+0000 7f87977fe640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f878407ea00 0x7f8784080e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.665+0000 7f87967fc640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87840762d0 0x7f8784078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f878c034000 tx=0x7f878c02e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.665+0000 7f87977fe640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f878407ea00 0x7f8784080e10 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.666+0000 7f877f7fe640 1 -- 172.21.15.52:0/1527254646 <== osd.7 v2:172.21.15.119:6816/4214692661 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8788011e50 con 0x7f878407ea00 2024-05-11T01:16:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.679+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 --> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f87980ced60 con 0x7f878407ea00 2024-05-11T01:16:20.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f877f7fe640 1 -- 172.21.15.52:0/1527254646 <== osd.7 v2:172.21.15.119:6816/4214692661 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f87980ced60 con 0x7f878407ea00 2024-05-11T01:16:20.680 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-11T01:16:20.680 INFO:teuthology.orchestra.run.smithi052.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T01:16:20.680 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-11T01:16:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f878407ea00 msgr2=0x7f8784080e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6816/4214692661,v1:172.21.15.119:6817/4214692661] conn(0x7f878407ea00 0x7f8784080e10 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87840762d0 msgr2=0x7f8784078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87840762d0 0x7f8784078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f878c034000 tx=0x7f878c02e000 comp rx=0 tx=0).stop 2024-05-11T01:16:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 msgr2=0x7f8798152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 --2- 172.21.15.52:0/1527254646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8798073f00 0x7f8798152500 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f878800ce30 tx=0x7f878800a650 comp rx=0 tx=0).stop 2024-05-11T01:16:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f87977fe640 1 -- 172.21.15.52:0/1527254646 reap_dead start 2024-05-11T01:16:20.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 shutdown_connections 2024-05-11T01:16:20.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 >> 172.21.15.52:0/1527254646 conn(0x7f87980cd7b0 msgr2=0x7f87980b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:20.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 shutdown_connections 2024-05-11T01:16:20.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.680+0000 7f879d7c8640 1 -- 172.21.15.52:0/1527254646 wait complete. 2024-05-11T01:16:20.691 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-11T01:16:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.796+0000 7efd23450640 1 Processor -- start 2024-05-11T01:16:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.796+0000 7efd23450640 1 -- start start 2024-05-11T01:16:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd23450640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd1c0d2ea0 0x7efd1c0d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd23450640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efd1c0b2600 0x7efd1c0b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd23450640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd1c0d1f90 con 0x7efd1c15f2d0 2024-05-11T01:16:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd23450640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7efd1c0d2270 con 0x7efd1c0b2600 2024-05-11T01:16:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd23450640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7efd1c0d2550 con 0x7efd1c0d2ea0 2024-05-11T01:16:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd219ed640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efd1c0b2600 0x7efd1c0b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd219ed640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7efd1c0b2600 0x7efd1c0b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43664/0 (socket says 172.21.15.52:43664) 2024-05-11T01:16:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd219ed640 1 -- 172.21.15.52:0/753790591 learned_addr learned my addr 172.21.15.52:0/753790591 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd209eb640 1 --2- 172.21.15.52:0/753790591 >> v2:172.21.15.119:3300/0 conn(0x7efd1c0d2ea0 0x7efd1c0d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd219ed640 1 -- 172.21.15.52:0/753790591 >> v2:172.21.15.119:3300/0 conn(0x7efd1c0d2ea0 msgr2=0x7efd1c0d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd219ed640 1 --2- 172.21.15.52:0/753790591 >> v2:172.21.15.119:3300/0 conn(0x7efd1c0d2ea0 0x7efd1c0d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd219ed640 1 -- 172.21.15.52:0/753790591 >> v1:172.21.15.119:6789/0 conn(0x7efd1c15f2d0 legacy=0x7efd1c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd219ed640 1 -- 172.21.15.52:0/753790591 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c0d2830 con 0x7efd1c0b2600 2024-05-11T01:16:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.797+0000 7efd209eb640 1 --2- 172.21.15.52:0/753790591 >> v2:172.21.15.119:3300/0 conn(0x7efd1c0d2ea0 0x7efd1c0d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.798+0000 7efd219ed640 1 --2- 172.21.15.52:0/753790591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 0x7efd1c0b2a10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0c9f90 tx=0x7efd1402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac80961faa0290a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.798+0000 7efd13fff640 1 -- 172.21.15.52:0/753790591 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14036070 con 0x7efd1c0b2600 2024-05-11T01:16:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.798+0000 7efd13fff640 1 -- 172.21.15.52:0/753790591 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd14031600 con 0x7efd1c0b2600 2024-05-11T01:16:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.798+0000 7efd13fff640 1 -- 172.21.15.52:0/753790591 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd140319d0 con 0x7efd1c0b2600 2024-05-11T01:16:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.799+0000 7efd23450640 1 -- 172.21.15.52:0/753790591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 msgr2=0x7efd1c0b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.799+0000 7efd23450640 1 --2- 172.21.15.52:0/753790591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 0x7efd1c0b2a10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0c9f90 tx=0x7efd1402a6a0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.799+0000 7efd23450640 1 -- 172.21.15.52:0/753790591 shutdown_connections 2024-05-11T01:16:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.799+0000 7efd23450640 1 --2- 172.21.15.52:0/753790591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 0x7efd1c0b2a10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.799+0000 7efd23450640 1 --2- 172.21.15.52:0/753790591 >> v2:172.21.15.119:3300/0 conn(0x7efd1c0d2ea0 0x7efd1c0d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.799+0000 7efd23450640 1 -- 172.21.15.52:0/753790591 >> 172.21.15.52:0/753790591 conn(0x7efd1c09ba20 msgr2=0x7efd1c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.799+0000 7efd23450640 1 -- 172.21.15.52:0/753790591 shutdown_connections 2024-05-11T01:16:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.799+0000 7efd23450640 1 -- 172.21.15.52:0/753790591 wait complete. 2024-05-11T01:16:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.800+0000 7efd23450640 1 Processor -- start 2024-05-11T01:16:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.800+0000 7efd23450640 1 -- start start 2024-05-11T01:16:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd23450640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 0x7efd1c1f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd23450640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd1c0d2ea0 0x7efd1c155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd23450640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd1c15f2d0 0x7efd1c1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd23450640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c15f830 con 0x7efd1c0b2600 2024-05-11T01:16:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd23450640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c15f6b0 con 0x7efd1c15f2d0 2024-05-11T01:16:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd23450640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c0d1b80 con 0x7efd1c0d2ea0 2024-05-11T01:16:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd211ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 0x7efd1c1f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd211ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 0x7efd1c1f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43678/0 (socket says 172.21.15.52:43678) 2024-05-11T01:16:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd211ec640 1 -- 172.21.15.52:0/1769711846 learned_addr learned my addr 172.21.15.52:0/1769711846 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd219ed640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd1c15f2d0 0x7efd1c1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd209eb640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd1c0d2ea0 0x7efd1c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd211ec640 1 -- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd1c0d2ea0 msgr2=0x7efd1c155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd211ec640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd1c0d2ea0 0x7efd1c155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd211ec640 1 -- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd1c15f2d0 msgr2=0x7efd1c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:20.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd211ec640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd1c15f2d0 0x7efd1c1614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:20.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd211ec640 1 -- 172.21.15.52:0/1769711846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c155940 con 0x7efd1c0b2600 2024-05-11T01:16:20.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd219ed640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd1c15f2d0 0x7efd1c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T01:16:20.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd211ec640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 0x7efd1c1f4550 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efd1800b3f0 tx=0x7efd1800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd11ffb640 1 -- 172.21.15.52:0/1769711846 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18015020 con 0x7efd1c0b2600 2024-05-11T01:16:20.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.801+0000 7efd11ffb640 1 -- 172.21.15.52:0/1769711846 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd18011830 con 0x7efd1c0b2600 2024-05-11T01:16:20.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.802+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c155c20 con 0x7efd1c0b2600 2024-05-11T01:16:20.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.802+0000 7efd11ffb640 1 -- 172.21.15.52:0/1769711846 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18011c20 con 0x7efd1c0b2600 2024-05-11T01:16:20.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.802+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c156930 con 0x7efd1c0b2600 2024-05-11T01:16:20.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.803+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1c069780 con 0x7efd1c0b2600 2024-05-11T01:16:20.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.803+0000 7efd11ffb640 1 -- 172.21.15.52:0/1769711846 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7efd18018020 con 0x7efd1c0b2600 2024-05-11T01:16:20.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.804+0000 7efd11ffb640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efd000762d0 0x7efd00078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:20.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.804+0000 7efd11ffb640 1 -- 172.21.15.52:0/1769711846 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7efd18097460 con 0x7efd1c0b2600 2024-05-11T01:16:20.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.804+0000 7efd209eb640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efd000762d0 0x7efd00078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:20.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.805+0000 7efd209eb640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efd000762d0 0x7efd00078790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd1c1f8130 tx=0x7efd0c008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:20.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:20.808+0000 7efd11ffb640 1 -- 172.21.15.52:0/1769711846 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efd1801f030 con 0x7efd1c0b2600 2024-05-11T01:16:21.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.002+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7efd1c0aff80 con 0x7efd000762d0 2024-05-11T01:16:21.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.003+0000 7efd11ffb640 1 -- 172.21.15.52:0/1769711846 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efd1c0aff80 con 0x7efd000762d0 2024-05-11T01:16:21.003 INFO:teuthology.orchestra.run.smithi052.stderr:instructed osd(s) 7 to deep-scrub 2024-05-11T01:16:21.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efd000762d0 msgr2=0x7efd00078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:21.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efd000762d0 0x7efd00078790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd1c1f8130 tx=0x7efd0c008040 comp rx=0 tx=0).stop 2024-05-11T01:16:21.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 msgr2=0x7efd1c1f4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:21.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 0x7efd1c1f4550 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efd1800b3f0 tx=0x7efd1800b8b0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 shutdown_connections 2024-05-11T01:16:21.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7efd000762d0 0x7efd00078790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd1c15f2d0 0x7efd1c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd1c0d2ea0 0x7efd1c155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 --2- 172.21.15.52:0/1769711846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c0b2600 0x7efd1c1f4550 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 >> 172.21.15.52:0/1769711846 conn(0x7efd1c09ba20 msgr2=0x7efd1c09be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:21.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 shutdown_connections 2024-05-11T01:16:21.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.005+0000 7efd23450640 1 -- 172.21.15.52:0/1769711846 wait complete. 2024-05-11T01:16:21.016 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:16:21.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.121+0000 7ff4456da640 1 Processor -- start 2024-05-11T01:16:21.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.121+0000 7ff4456da640 1 -- start start 2024-05-11T01:16:21.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.121+0000 7ff4456da640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff4400d2ea0 0x7ff4400d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:21.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.121+0000 7ff4456da640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4400b2600 0x7ff4400b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:21.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.121+0000 7ff4456da640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4400d1f90 con 0x7ff44015f2d0 2024-05-11T01:16:21.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.121+0000 7ff4456da640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7ff4400d2270 con 0x7ff4400d2ea0 2024-05-11T01:16:21.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.121+0000 7ff4456da640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7ff4400d2550 con 0x7ff4400b2600 2024-05-11T01:16:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.122+0000 7ff43e7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff4400d2ea0 0x7ff4400d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.122+0000 7ff43e7fc640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7ff4400d2ea0 0x7ff4400d3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43682/0 (socket says 172.21.15.52:43682) 2024-05-11T01:16:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.122+0000 7ff43e7fc640 1 -- 172.21.15.52:0/652214995 learned_addr learned my addr 172.21.15.52:0/652214995 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.122+0000 7ff43f7fe640 1 --2- 172.21.15.52:0/652214995 >> v2:172.21.15.119:3300/0 conn(0x7ff4400b2600 0x7ff4400b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.122+0000 7ff43e7fc640 1 -- 172.21.15.52:0/652214995 >> v2:172.21.15.119:3300/0 conn(0x7ff4400b2600 msgr2=0x7ff4400b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.122+0000 7ff43e7fc640 1 --2- 172.21.15.52:0/652214995 >> v2:172.21.15.119:3300/0 conn(0x7ff4400b2600 0x7ff4400b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.122+0000 7ff43e7fc640 1 -- 172.21.15.52:0/652214995 >> v1:172.21.15.119:6789/0 conn(0x7ff44015f2d0 legacy=0x7ff44006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.122+0000 7ff43e7fc640 1 -- 172.21.15.52:0/652214995 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4400d2830 con 0x7ff4400d2ea0 2024-05-11T01:16:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.122+0000 7ff43f7fe640 1 --2- 172.21.15.52:0/652214995 >> v2:172.21.15.119:3300/0 conn(0x7ff4400b2600 0x7ff4400b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff43e7fc640 1 --2- 172.21.15.52:0/652214995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 0x7ff4400d3330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff44006f600 tx=0x7ff42802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b60aa6ee13e7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff43dffb640 1 -- 172.21.15.52:0/652214995 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428037070 con 0x7ff4400d2ea0 2024-05-11T01:16:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff43dffb640 1 -- 172.21.15.52:0/652214995 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff428032440 con 0x7ff4400d2ea0 2024-05-11T01:16:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff43dffb640 1 -- 172.21.15.52:0/652214995 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428032810 con 0x7ff4400d2ea0 2024-05-11T01:16:21.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff4456da640 1 -- 172.21.15.52:0/652214995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 msgr2=0x7ff4400d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:21.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff4456da640 1 --2- 172.21.15.52:0/652214995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 0x7ff4400d3330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff44006f600 tx=0x7ff42802a1e0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff4456da640 1 -- 172.21.15.52:0/652214995 shutdown_connections 2024-05-11T01:16:21.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff4456da640 1 --2- 172.21.15.52:0/652214995 >> v2:172.21.15.119:3300/0 conn(0x7ff4400b2600 0x7ff4400b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff4456da640 1 --2- 172.21.15.52:0/652214995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 0x7ff4400d3330 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff4456da640 1 -- 172.21.15.52:0/652214995 >> 172.21.15.52:0/652214995 conn(0x7ff44009ba20 msgr2=0x7ff44009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:21.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff4456da640 1 -- 172.21.15.52:0/652214995 shutdown_connections 2024-05-11T01:16:21.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.123+0000 7ff4456da640 1 -- 172.21.15.52:0/652214995 wait complete. 2024-05-11T01:16:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.125+0000 7ff4456da640 1 Processor -- start 2024-05-11T01:16:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.125+0000 7ff4456da640 1 -- start start 2024-05-11T01:16:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.125+0000 7ff4456da640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4400b2600 0x7ff440154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.125+0000 7ff4456da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 0x7ff4401564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.125+0000 7ff4456da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44015f2d0 0x7ff440156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff4456da640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff44015f830 con 0x7ff4400d2ea0 2024-05-11T01:16:21.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff4456da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff44015f6b0 con 0x7ff44015f2d0 2024-05-11T01:16:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff4456da640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7ff4400d1b80 con 0x7ff4400b2600 2024-05-11T01:16:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 0x7ff4401564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 0x7ff4401564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43684/0 (socket says 172.21.15.52:43684) 2024-05-11T01:16:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43e7fc640 1 -- 172.21.15.52:0/4136475150 learned_addr learned my addr 172.21.15.52:0/4136475150 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43f7fe640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44015f2d0 0x7ff440156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43effd640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4400b2600 0x7ff440154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:21.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43e7fc640 1 -- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4400b2600 msgr2=0x7ff440154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43e7fc640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4400b2600 0x7ff440154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43e7fc640 1 -- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44015f2d0 msgr2=0x7ff440156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43e7fc640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44015f2d0 0x7ff440156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43e7fc640 1 -- 172.21.15.52:0/4136475150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff440155420 con 0x7ff4400d2ea0 2024-05-11T01:16:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43effd640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4400b2600 0x7ff440154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff43e7fc640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 0x7ff4401564c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff428000c00 tx=0x7ff4280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff4377fe640 1 -- 172.21.15.52:0/4136475150 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428041020 con 0x7ff4400d2ea0 2024-05-11T01:16:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff4377fe640 1 -- 172.21.15.52:0/4136475150 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff428030970 con 0x7ff4400d2ea0 2024-05-11T01:16:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff440155700 con 0x7ff4400d2ea0 2024-05-11T01:16:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff4377fe640 1 -- 172.21.15.52:0/4136475150 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428037040 con 0x7ff4400d2ea0 2024-05-11T01:16:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.126+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff440155cc0 con 0x7ff4400d2ea0 2024-05-11T01:16:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.127+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff440069780 con 0x7ff4400d2ea0 2024-05-11T01:16:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.128+0000 7ff4377fe640 1 -- 172.21.15.52:0/4136475150 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7ff428044020 con 0x7ff4400d2ea0 2024-05-11T01:16:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.129+0000 7ff4377fe640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff4200762d0 0x7ff420078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:21.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.129+0000 7ff4377fe640 1 -- 172.21.15.52:0/4136475150 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7ff4280b8260 con 0x7ff4400d2ea0 2024-05-11T01:16:21.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.129+0000 7ff43effd640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff4200762d0 0x7ff420078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:21.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.129+0000 7ff43effd640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff4200762d0 0x7ff420078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff430007e90 tx=0x7ff4300073a0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:21.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.134+0000 7ff4377fe640 1 -- 172.21.15.52:0/4136475150 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff42804b030 con 0x7ff4400d2ea0 2024-05-11T01:16:21.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.321+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4400afda0 con 0x7ff4200762d0 2024-05-11T01:16:21.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.328+0000 7ff4377fe640 1 -- 172.21.15.52:0/4136475150 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251840 (secure 0 0 0) 0x7ff4400afda0 con 0x7ff4200762d0 2024-05-11T01:16:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:16:21.332 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:16:21.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff4200762d0 msgr2=0x7ff420078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:21.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff4200762d0 0x7ff420078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff430007e90 tx=0x7ff4300073a0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 msgr2=0x7ff4401564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:21.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 0x7ff4401564c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff428000c00 tx=0x7ff4280046c0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 shutdown_connections 2024-05-11T01:16:21.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7ff4200762d0 0x7ff420078790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44015f2d0 0x7ff440156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4400d2ea0 0x7ff4401564c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 --2- 172.21.15.52:0/4136475150 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff4400b2600 0x7ff440154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:21.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 >> 172.21.15.52:0/4136475150 conn(0x7ff44009ba20 msgr2=0x7ff44009be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:21.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 shutdown_connections 2024-05-11T01:16:21.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:21.331+0000 7ff4456da640 1 -- 172.21.15.52:0/4136475150 wait complete. 2024-05-11T01:16:21.342 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-05-11T01:16:19.741418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594175,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":174,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317588,"kb_used_data":7080,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754657132,"statfs":{"total":773094113280,"available":772768903168,"internally_reserved":0,"allocated":7249920,"data_stored":4059958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":317836849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003265"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854522+0000","last_change":"2024-05-11T01:13:59.434893+0000","last_active":"2024-05-11T01:14:46.854522+0000","last_peered":"2024-05-11T01:14:46.854522+0000","last_clean":"2024-05-11T01:14:46.854522+0000","last_became_active":"2024-05-11T01:13:59.434594+0000","last_became_peered":"2024-05-11T01:13:59.434594+0000","last_unstale":"2024-05-11T01:14:46.854522+0000","last_undegraded":"2024-05-11T01:14:46.854522+0000","last_fullsized":"2024-05-11T01:14:46.854522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:49:33.685964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854143+0000","last_change":"2024-05-11T01:13:59.446282+0000","last_active":"2024-05-11T01:14:46.854143+0000","last_peered":"2024-05-11T01:14:46.854143+0000","last_clean":"2024-05-11T01:14:46.854143+0000","last_became_active":"2024-05-11T01:13:59.445971+0000","last_became_peered":"2024-05-11T01:13:59.445971+0000","last_unstale":"2024-05-11T01:14:46.854143+0000","last_undegraded":"2024-05-11T01:14:46.854143+0000","last_fullsized":"2024-05-11T01:14:46.854143+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:50.869559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839269+0000","last_change":"2024-05-11T01:13:59.430661+0000","last_active":"2024-05-11T01:14:46.839269+0000","last_peered":"2024-05-11T01:14:46.839269+0000","last_clean":"2024-05-11T01:14:46.839269+0000","last_became_active":"2024-05-11T01:13:59.430408+0000","last_became_peered":"2024-05-11T01:13:59.430408+0000","last_unstale":"2024-05-11T01:14:46.839269+0000","last_undegraded":"2024-05-11T01:14:46.839269+0000","last_fullsized":"2024-05-11T01:14:46.839269+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:14:03.379607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854537+0000","last_change":"2024-05-11T01:13:59.448424+0000","last_active":"2024-05-11T01:14:46.854537+0000","last_peered":"2024-05-11T01:14:46.854537+0000","last_clean":"2024-05-11T01:14:46.854537+0000","last_became_active":"2024-05-11T01:13:59.446986+0000","last_became_peered":"2024-05-11T01:13:59.446986+0000","last_unstale":"2024-05-11T01:14:46.854537+0000","last_undegraded":"2024-05-11T01:14:46.854537+0000","last_fullsized":"2024-05-11T01:14:46.854537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853610+0000","last_change":"2024-05-11T01:13:59.446369+0000","last_active":"2024-05-11T01:14:46.853610+0000","last_peered":"2024-05-11T01:14:46.853610+0000","last_clean":"2024-05-11T01:14:46.853610+0000","last_became_active":"2024-05-11T01:13:59.445922+0000","last_became_peered":"2024-05-11T01:13:59.445922+0000","last_unstale":"2024-05-11T01:14:46.853610+0000","last_undegraded":"2024-05-11T01:14:46.853610+0000","last_fullsized":"2024-05-11T01:14:46.853610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:15.278361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837202+0000","last_change":"2024-05-11T01:13:59.435535+0000","last_active":"2024-05-11T01:14:46.837202+0000","last_peered":"2024-05-11T01:14:46.837202+0000","last_clean":"2024-05-11T01:14:46.837202+0000","last_became_active":"2024-05-11T01:13:59.434634+0000","last_became_peered":"2024-05-11T01:13:59.434634+0000","last_unstale":"2024-05-11T01:14:46.837202+0000","last_undegraded":"2024-05-11T01:14:46.837202+0000","last_fullsized":"2024-05-11T01:14:46.837202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:34:16.451030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854001+0000","last_change":"2024-05-11T01:13:59.434552+0000","last_active":"2024-05-11T01:14:46.854001+0000","last_peered":"2024-05-11T01:14:46.854001+0000","last_clean":"2024-05-11T01:14:46.854001+0000","last_became_active":"2024-05-11T01:13:59.433976+0000","last_became_peered":"2024-05-11T01:13:59.433976+0000","last_unstale":"2024-05-11T01:14:46.854001+0000","last_undegraded":"2024-05-11T01:14:46.854001+0000","last_fullsized":"2024-05-11T01:14:46.854001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853803+0000","last_change":"2024-05-11T01:13:59.447658+0000","last_active":"2024-05-11T01:14:46.853803+0000","last_peered":"2024-05-11T01:14:46.853803+0000","last_clean":"2024-05-11T01:14:46.853803+0000","last_became_active":"2024-05-11T01:13:59.446507+0000","last_became_peered":"2024-05-11T01:13:59.446507+0000","last_unstale":"2024-05-11T01:14:46.853803+0000","last_undegraded":"2024-05-11T01:14:46.853803+0000","last_fullsized":"2024-05-11T01:14:46.853803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:34:51.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854683+0000","last_change":"2024-05-11T01:13:59.449973+0000","last_active":"2024-05-11T01:14:46.854683+0000","last_peered":"2024-05-11T01:14:46.854683+0000","last_clean":"2024-05-11T01:14:46.854683+0000","last_became_active":"2024-05-11T01:13:59.449670+0000","last_became_peered":"2024-05-11T01:13:59.449670+0000","last_unstale":"2024-05-11T01:14:46.854683+0000","last_undegraded":"2024-05-11T01:14:46.854683+0000","last_fullsized":"2024-05-11T01:14:46.854683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854986+0000","last_change":"2024-05-11T01:13:59.450134+0000","last_active":"2024-05-11T01:14:46.854986+0000","last_peered":"2024-05-11T01:14:46.854986+0000","last_clean":"2024-05-11T01:14:46.854986+0000","last_became_active":"2024-05-11T01:13:59.449822+0000","last_became_peered":"2024-05-11T01:13:59.449822+0000","last_unstale":"2024-05-11T01:14:46.854986+0000","last_undegraded":"2024-05-11T01:14:46.854986+0000","last_fullsized":"2024-05-11T01:14:46.854986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.840088+0000","last_change":"2024-05-11T01:14:46.840088+0000","last_active":"2024-05-11T01:14:46.840088+0000","last_peered":"2024-05-11T01:14:46.840088+0000","last_clean":"2024-05-11T01:14:46.840088+0000","last_became_active":"2024-05-11T01:14:46.839837+0000","last_became_peered":"2024-05-11T01:14:46.839837+0000","last_unstale":"2024-05-11T01:14:46.840088+0000","last_undegraded":"2024-05-11T01:14:46.840088+0000","last_fullsized":"2024-05-11T01:14:46.840088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:38:11.978713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853737+0000","last_change":"2024-05-11T01:13:59.428513+0000","last_active":"2024-05-11T01:14:46.853737+0000","last_peered":"2024-05-11T01:14:46.853737+0000","last_clean":"2024-05-11T01:14:46.853737+0000","last_became_active":"2024-05-11T01:13:59.428202+0000","last_became_peered":"2024-05-11T01:13:59.428202+0000","last_unstale":"2024-05-11T01:14:46.853737+0000","last_undegraded":"2024-05-11T01:14:46.853737+0000","last_fullsized":"2024-05-11T01:14:46.853737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:23.500916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854223+0000","last_change":"2024-05-11T01:13:59.435402+0000","last_active":"2024-05-11T01:14:46.854223+0000","last_peered":"2024-05-11T01:14:46.854223+0000","last_clean":"2024-05-11T01:14:46.854223+0000","last_became_active":"2024-05-11T01:13:59.433324+0000","last_became_peered":"2024-05-11T01:13:59.433324+0000","last_unstale":"2024-05-11T01:14:46.854223+0000","last_undegraded":"2024-05-11T01:14:46.854223+0000","last_fullsized":"2024-05-11T01:14:46.854223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:36:46.612605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854315+0000","last_change":"2024-05-11T01:13:59.447113+0000","last_active":"2024-05-11T01:14:46.854315+0000","last_peered":"2024-05-11T01:14:46.854315+0000","last_clean":"2024-05-11T01:14:46.854315+0000","last_became_active":"2024-05-11T01:13:59.446349+0000","last_became_peered":"2024-05-11T01:13:59.446349+0000","last_unstale":"2024-05-11T01:14:46.854315+0000","last_undegraded":"2024-05-11T01:14:46.854315+0000","last_fullsized":"2024-05-11T01:14:46.854315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836869+0000","last_change":"2024-05-11T01:13:59.437811+0000","last_active":"2024-05-11T01:14:46.836869+0000","last_peered":"2024-05-11T01:14:46.836869+0000","last_clean":"2024-05-11T01:14:46.836869+0000","last_became_active":"2024-05-11T01:13:59.427975+0000","last_became_peered":"2024-05-11T01:13:59.427975+0000","last_unstale":"2024-05-11T01:14:46.836869+0000","last_undegraded":"2024-05-11T01:14:46.836869+0000","last_fullsized":"2024-05-11T01:14:46.836869+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:05:27.108182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836920+0000","last_change":"2024-05-11T01:13:59.446666+0000","last_active":"2024-05-11T01:14:46.836920+0000","last_peered":"2024-05-11T01:14:46.836920+0000","last_clean":"2024-05-11T01:14:46.836920+0000","last_became_active":"2024-05-11T01:13:59.446396+0000","last_became_peered":"2024-05-11T01:13:59.446396+0000","last_unstale":"2024-05-11T01:14:46.836920+0000","last_undegraded":"2024-05-11T01:14:46.836920+0000","last_fullsized":"2024-05-11T01:14:46.836920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:55:23.003515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839156+0000","last_change":"2024-05-11T01:13:59.437261+0000","last_active":"2024-05-11T01:14:46.839156+0000","last_peered":"2024-05-11T01:14:46.839156+0000","last_clean":"2024-05-11T01:14:46.839156+0000","last_became_active":"2024-05-11T01:13:59.436992+0000","last_became_peered":"2024-05-11T01:13:59.436992+0000","last_unstale":"2024-05-11T01:14:46.839156+0000","last_undegraded":"2024-05-11T01:14:46.839156+0000","last_fullsized":"2024-05-11T01:14:46.839156+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:04:53.116569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839026+0000","last_change":"2024-05-11T01:13:59.445508+0000","last_active":"2024-05-11T01:14:46.839026+0000","last_peered":"2024-05-11T01:14:46.839026+0000","last_clean":"2024-05-11T01:14:46.839026+0000","last_became_active":"2024-05-11T01:13:59.445318+0000","last_became_peered":"2024-05-11T01:13:59.445318+0000","last_unstale":"2024-05-11T01:14:46.839026+0000","last_undegraded":"2024-05-11T01:14:46.839026+0000","last_fullsized":"2024-05-11T01:14:46.839026+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:21.154101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.856442+0000","last_change":"2024-05-11T01:14:46.855679+0000","last_active":"2024-05-11T01:14:46.856442+0000","last_peered":"2024-05-11T01:14:46.856442+0000","last_clean":"2024-05-11T01:14:46.856442+0000","last_became_active":"2024-05-11T01:14:46.855320+0000","last_became_peered":"2024-05-11T01:14:46.855320+0000","last_unstale":"2024-05-11T01:14:46.856442+0000","last_undegraded":"2024-05-11T01:14:46.856442+0000","last_fullsized":"2024-05-11T01:14:46.856442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:48:36.304487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854586+0000","last_change":"2024-05-11T01:13:59.444231+0000","last_active":"2024-05-11T01:14:46.854586+0000","last_peered":"2024-05-11T01:14:46.854586+0000","last_clean":"2024-05-11T01:14:46.854586+0000","last_became_active":"2024-05-11T01:13:59.443762+0000","last_became_peered":"2024-05-11T01:13:59.443762+0000","last_unstale":"2024-05-11T01:14:46.854586+0000","last_undegraded":"2024-05-11T01:14:46.854586+0000","last_fullsized":"2024-05-11T01:14:46.854586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:25:17.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837322+0000","last_change":"2024-05-11T01:13:59.437781+0000","last_active":"2024-05-11T01:14:46.837322+0000","last_peered":"2024-05-11T01:14:46.837322+0000","last_clean":"2024-05-11T01:14:46.837322+0000","last_became_active":"2024-05-11T01:13:59.437444+0000","last_became_peered":"2024-05-11T01:13:59.437444+0000","last_unstale":"2024-05-11T01:14:46.837322+0000","last_undegraded":"2024-05-11T01:14:46.837322+0000","last_fullsized":"2024-05-11T01:14:46.837322+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:31:12.326102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854523+0000","last_change":"2024-05-11T01:13:59.447343+0000","last_active":"2024-05-11T01:14:46.854523+0000","last_peered":"2024-05-11T01:14:46.854523+0000","last_clean":"2024-05-11T01:14:46.854523+0000","last_became_active":"2024-05-11T01:13:59.446779+0000","last_became_peered":"2024-05-11T01:13:59.446779+0000","last_unstale":"2024-05-11T01:14:46.854523+0000","last_undegraded":"2024-05-11T01:14:46.854523+0000","last_fullsized":"2024-05-11T01:14:46.854523+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837263+0000","last_change":"2024-05-11T01:13:59.431582+0000","last_active":"2024-05-11T01:14:46.837263+0000","last_peered":"2024-05-11T01:14:46.837263+0000","last_clean":"2024-05-11T01:14:46.837263+0000","last_became_active":"2024-05-11T01:13:59.431309+0000","last_became_peered":"2024-05-11T01:13:59.431309+0000","last_unstale":"2024-05-11T01:14:46.837263+0000","last_undegraded":"2024-05-11T01:14:46.837263+0000","last_fullsized":"2024-05-11T01:14:46.837263+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837032+0000","last_change":"2024-05-11T01:13:59.433902+0000","last_active":"2024-05-11T01:14:46.837032+0000","last_peered":"2024-05-11T01:14:46.837032+0000","last_clean":"2024-05-11T01:14:46.837032+0000","last_became_active":"2024-05-11T01:13:59.433606+0000","last_became_peered":"2024-05-11T01:13:59.433606+0000","last_unstale":"2024-05-11T01:14:46.837032+0000","last_undegraded":"2024-05-11T01:14:46.837032+0000","last_fullsized":"2024-05-11T01:14:46.837032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837537+0000","last_change":"2024-05-11T01:13:59.436026+0000","last_active":"2024-05-11T01:14:46.837537+0000","last_peered":"2024-05-11T01:14:46.837537+0000","last_clean":"2024-05-11T01:14:46.837537+0000","last_became_active":"2024-05-11T01:13:59.435708+0000","last_became_peered":"2024-05-11T01:13:59.435708+0000","last_unstale":"2024-05-11T01:14:46.837537+0000","last_undegraded":"2024-05-11T01:14:46.837537+0000","last_fullsized":"2024-05-11T01:14:46.837537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:04:20.616078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854605+0000","last_change":"2024-05-11T01:13:59.441132+0000","last_active":"2024-05-11T01:14:46.854605+0000","last_peered":"2024-05-11T01:14:46.854605+0000","last_clean":"2024-05-11T01:14:46.854605+0000","last_became_active":"2024-05-11T01:13:59.440852+0000","last_became_peered":"2024-05-11T01:13:59.440852+0000","last_unstale":"2024-05-11T01:14:46.854605+0000","last_undegraded":"2024-05-11T01:14:46.854605+0000","last_fullsized":"2024-05-11T01:14:46.854605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:18:58.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839326+0000","last_change":"2024-05-11T01:13:59.436547+0000","last_active":"2024-05-11T01:14:46.839326+0000","last_peered":"2024-05-11T01:14:46.839326+0000","last_clean":"2024-05-11T01:14:46.839326+0000","last_became_active":"2024-05-11T01:13:59.436260+0000","last_became_peered":"2024-05-11T01:13:59.436260+0000","last_unstale":"2024-05-11T01:14:46.839326+0000","last_undegraded":"2024-05-11T01:14:46.839326+0000","last_fullsized":"2024-05-11T01:14:46.839326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:27.176219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853867+0000","last_change":"2024-05-11T01:13:59.425177+0000","last_active":"2024-05-11T01:14:46.853867+0000","last_peered":"2024-05-11T01:14:46.853867+0000","last_clean":"2024-05-11T01:14:46.853867+0000","last_became_active":"2024-05-11T01:13:59.424328+0000","last_became_peered":"2024-05-11T01:13:59.424328+0000","last_unstale":"2024-05-11T01:14:46.853867+0000","last_undegraded":"2024-05-11T01:14:46.853867+0000","last_fullsized":"2024-05-11T01:14:46.853867+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:29.739616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837190+0000","last_change":"2024-05-11T01:13:59.436551+0000","last_active":"2024-05-11T01:14:46.837190+0000","last_peered":"2024-05-11T01:14:46.837190+0000","last_clean":"2024-05-11T01:14:46.837190+0000","last_became_active":"2024-05-11T01:13:59.436257+0000","last_became_peered":"2024-05-11T01:13:59.436257+0000","last_unstale":"2024-05-11T01:14:46.837190+0000","last_undegraded":"2024-05-11T01:14:46.837190+0000","last_fullsized":"2024-05-11T01:14:46.837190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:37:44.925797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838948+0000","last_change":"2024-05-11T01:13:59.435398+0000","last_active":"2024-05-11T01:14:46.838948+0000","last_peered":"2024-05-11T01:14:46.838948+0000","last_clean":"2024-05-11T01:14:46.838948+0000","last_became_active":"2024-05-11T01:13:59.435148+0000","last_became_peered":"2024-05-11T01:13:59.435148+0000","last_unstale":"2024-05-11T01:14:46.838948+0000","last_undegraded":"2024-05-11T01:14:46.838948+0000","last_fullsized":"2024-05-11T01:14:46.838948+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:14:43.435242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837071+0000","last_change":"2024-05-11T01:13:59.438524+0000","last_active":"2024-05-11T01:14:46.837071+0000","last_peered":"2024-05-11T01:14:46.837071+0000","last_clean":"2024-05-11T01:14:46.837071+0000","last_became_active":"2024-05-11T01:13:59.438262+0000","last_became_peered":"2024-05-11T01:13:59.438262+0000","last_unstale":"2024-05-11T01:14:46.837071+0000","last_undegraded":"2024-05-11T01:14:46.837071+0000","last_fullsized":"2024-05-11T01:14:46.837071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:44:46.930438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854377+0000","last_change":"2024-05-11T01:13:59.432090+0000","last_active":"2024-05-11T01:14:46.854377+0000","last_peered":"2024-05-11T01:14:46.854377+0000","last_clean":"2024-05-11T01:14:46.854377+0000","last_became_active":"2024-05-11T01:13:59.431691+0000","last_became_peered":"2024-05-11T01:13:59.431691+0000","last_unstale":"2024-05-11T01:14:46.854377+0000","last_undegraded":"2024-05-11T01:14:46.854377+0000","last_fullsized":"2024-05-11T01:14:46.854377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:11:10.857350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836734+0000","last_change":"2024-05-11T01:13:59.440442+0000","last_active":"2024-05-11T01:14:46.836734+0000","last_peered":"2024-05-11T01:14:46.836734+0000","last_clean":"2024-05-11T01:14:46.836734+0000","last_became_active":"2024-05-11T01:13:59.440107+0000","last_became_peered":"2024-05-11T01:13:59.440107+0000","last_unstale":"2024-05-11T01:14:46.836734+0000","last_undegraded":"2024-05-11T01:14:46.836734+0000","last_fullsized":"2024-05-11T01:14:46.836734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:58:40.948977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837590+0000","last_change":"2024-05-11T01:13:58.440158+0000","last_active":"2024-05-11T01:14:46.837590+0000","last_peered":"2024-05-11T01:14:46.837590+0000","last_clean":"2024-05-11T01:14:46.837590+0000","last_became_active":"2024-05-11T01:13:58.439841+0000","last_became_peered":"2024-05-11T01:13:58.439841+0000","last_unstale":"2024-05-11T01:14:46.837590+0000","last_undegraded":"2024-05-11T01:14:46.837590+0000","last_fullsized":"2024-05-11T01:14:46.837590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837594+0000","last_change":"2024-05-11T01:13:59.433723+0000","last_active":"2024-05-11T01:14:46.837594+0000","last_peered":"2024-05-11T01:14:46.837594+0000","last_clean":"2024-05-11T01:14:46.837594+0000","last_became_active":"2024-05-11T01:13:59.433441+0000","last_became_peered":"2024-05-11T01:13:59.433441+0000","last_unstale":"2024-05-11T01:14:46.837594+0000","last_undegraded":"2024-05-11T01:14:46.837594+0000","last_fullsized":"2024-05-11T01:14:46.837594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839249+0000","last_change":"2024-05-11T01:13:58.433974+0000","last_active":"2024-05-11T01:14:46.839249+0000","last_peered":"2024-05-11T01:14:46.839249+0000","last_clean":"2024-05-11T01:14:46.839249+0000","last_became_active":"2024-05-11T01:13:58.433649+0000","last_became_peered":"2024-05-11T01:13:58.433649+0000","last_unstale":"2024-05-11T01:14:46.839249+0000","last_undegraded":"2024-05-11T01:14:46.839249+0000","last_fullsized":"2024-05-11T01:14:46.839249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:59.322532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853811+0000","last_change":"2024-05-11T01:13:59.439364+0000","last_active":"2024-05-11T01:14:46.853811+0000","last_peered":"2024-05-11T01:14:46.853811+0000","last_clean":"2024-05-11T01:14:46.853811+0000","last_became_active":"2024-05-11T01:13:59.439073+0000","last_became_peered":"2024-05-11T01:13:59.439073+0000","last_unstale":"2024-05-11T01:14:46.853811+0000","last_undegraded":"2024-05-11T01:14:46.853811+0000","last_fullsized":"2024-05-11T01:14:46.853811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:48.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837019+0000","last_change":"2024-05-11T01:13:58.435449+0000","last_active":"2024-05-11T01:14:46.837019+0000","last_peered":"2024-05-11T01:14:46.837019+0000","last_clean":"2024-05-11T01:14:46.837019+0000","last_became_active":"2024-05-11T01:13:58.434345+0000","last_became_peered":"2024-05-11T01:13:58.434345+0000","last_unstale":"2024-05-11T01:14:46.837019+0000","last_undegraded":"2024-05-11T01:14:46.837019+0000","last_fullsized":"2024-05-11T01:14:46.837019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:08:58.718268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836974+0000","last_change":"2024-05-11T01:13:59.440315+0000","last_active":"2024-05-11T01:14:46.836974+0000","last_peered":"2024-05-11T01:14:46.836974+0000","last_clean":"2024-05-11T01:14:46.836974+0000","last_became_active":"2024-05-11T01:13:59.439153+0000","last_became_peered":"2024-05-11T01:13:59.439153+0000","last_unstale":"2024-05-11T01:14:46.836974+0000","last_undegraded":"2024-05-11T01:14:46.836974+0000","last_fullsized":"2024-05-11T01:14:46.836974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:38:04.924508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854374+0000","last_change":"2024-05-11T01:13:58.434357+0000","last_active":"2024-05-11T01:14:46.854374+0000","last_peered":"2024-05-11T01:14:46.854374+0000","last_clean":"2024-05-11T01:14:46.854374+0000","last_became_active":"2024-05-11T01:13:58.433392+0000","last_became_peered":"2024-05-11T01:13:58.433392+0000","last_unstale":"2024-05-11T01:14:46.854374+0000","last_undegraded":"2024-05-11T01:14:46.854374+0000","last_fullsized":"2024-05-11T01:14:46.854374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:05:14.276107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838856+0000","last_change":"2024-05-11T01:13:59.435145+0000","last_active":"2024-05-11T01:14:46.838856+0000","last_peered":"2024-05-11T01:14:46.838856+0000","last_clean":"2024-05-11T01:14:46.838856+0000","last_became_active":"2024-05-11T01:13:59.434862+0000","last_became_peered":"2024-05-11T01:13:59.434862+0000","last_unstale":"2024-05-11T01:14:46.838856+0000","last_undegraded":"2024-05-11T01:14:46.838856+0000","last_fullsized":"2024-05-11T01:14:46.838856+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:17.395853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837423+0000","last_change":"2024-05-11T01:13:58.439588+0000","last_active":"2024-05-11T01:14:46.837423+0000","last_peered":"2024-05-11T01:14:46.837423+0000","last_clean":"2024-05-11T01:14:46.837423+0000","last_became_active":"2024-05-11T01:13:58.439133+0000","last_became_peered":"2024-05-11T01:13:58.439133+0000","last_unstale":"2024-05-11T01:14:46.837423+0000","last_undegraded":"2024-05-11T01:14:46.837423+0000","last_fullsized":"2024-05-11T01:14:46.837423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854573+0000","last_change":"2024-05-11T01:13:59.448882+0000","last_active":"2024-05-11T01:14:46.854573+0000","last_peered":"2024-05-11T01:14:46.854573+0000","last_clean":"2024-05-11T01:14:46.854573+0000","last_became_active":"2024-05-11T01:13:59.448208+0000","last_became_peered":"2024-05-11T01:13:59.448208+0000","last_unstale":"2024-05-11T01:14:46.854573+0000","last_undegraded":"2024-05-11T01:14:46.854573+0000","last_fullsized":"2024-05-11T01:14:46.854573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:11:19.642694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839012+0000","last_change":"2024-05-11T01:13:58.428838+0000","last_active":"2024-05-11T01:14:46.839012+0000","last_peered":"2024-05-11T01:14:46.839012+0000","last_clean":"2024-05-11T01:14:46.839012+0000","last_became_active":"2024-05-11T01:13:58.428556+0000","last_became_peered":"2024-05-11T01:13:58.428556+0000","last_unstale":"2024-05-11T01:14:46.839012+0000","last_undegraded":"2024-05-11T01:14:46.839012+0000","last_fullsized":"2024-05-11T01:14:46.839012+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:01:35.971417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854781+0000","last_change":"2024-05-11T01:13:59.447010+0000","last_active":"2024-05-11T01:14:46.854781+0000","last_peered":"2024-05-11T01:14:46.854781+0000","last_clean":"2024-05-11T01:14:46.854781+0000","last_became_active":"2024-05-11T01:13:59.446713+0000","last_became_peered":"2024-05-11T01:13:59.446713+0000","last_unstale":"2024-05-11T01:14:46.854781+0000","last_undegraded":"2024-05-11T01:14:46.854781+0000","last_fullsized":"2024-05-11T01:14:46.854781+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:03:04.844818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853638+0000","last_change":"2024-05-11T01:13:58.428031+0000","last_active":"2024-05-11T01:14:46.853638+0000","last_peered":"2024-05-11T01:14:46.853638+0000","last_clean":"2024-05-11T01:14:46.853638+0000","last_became_active":"2024-05-11T01:13:58.427742+0000","last_became_peered":"2024-05-11T01:13:58.427742+0000","last_unstale":"2024-05-11T01:14:46.853638+0000","last_undegraded":"2024-05-11T01:14:46.853638+0000","last_fullsized":"2024-05-11T01:14:46.853638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836923+0000","last_change":"2024-05-11T01:13:59.431589+0000","last_active":"2024-05-11T01:14:46.836923+0000","last_peered":"2024-05-11T01:14:46.836923+0000","last_clean":"2024-05-11T01:14:46.836923+0000","last_became_active":"2024-05-11T01:13:59.431292+0000","last_became_peered":"2024-05-11T01:13:59.431292+0000","last_unstale":"2024-05-11T01:14:46.836923+0000","last_undegraded":"2024-05-11T01:14:46.836923+0000","last_fullsized":"2024-05-11T01:14:46.836923+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853944+0000","last_change":"2024-05-11T01:13:58.436246+0000","last_active":"2024-05-11T01:14:46.853944+0000","last_peered":"2024-05-11T01:14:46.853944+0000","last_clean":"2024-05-11T01:14:46.853944+0000","last_became_active":"2024-05-11T01:13:58.435967+0000","last_became_peered":"2024-05-11T01:13:58.435967+0000","last_unstale":"2024-05-11T01:14:46.853944+0000","last_undegraded":"2024-05-11T01:14:46.853944+0000","last_fullsized":"2024-05-11T01:14:46.853944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:22.242555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839175+0000","last_change":"2024-05-11T01:13:59.445506+0000","last_active":"2024-05-11T01:14:46.839175+0000","last_peered":"2024-05-11T01:14:46.839175+0000","last_clean":"2024-05-11T01:14:46.839175+0000","last_became_active":"2024-05-11T01:13:59.445232+0000","last_became_peered":"2024-05-11T01:13:59.445232+0000","last_unstale":"2024-05-11T01:14:46.839175+0000","last_undegraded":"2024-05-11T01:14:46.839175+0000","last_fullsized":"2024-05-11T01:14:46.839175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:18:27.371597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839300+0000","last_change":"2024-05-11T01:13:58.426187+0000","last_active":"2024-05-11T01:14:46.839300+0000","last_peered":"2024-05-11T01:14:46.839300+0000","last_clean":"2024-05-11T01:14:46.839300+0000","last_became_active":"2024-05-11T01:13:58.425913+0000","last_became_peered":"2024-05-11T01:13:58.425913+0000","last_unstale":"2024-05-11T01:14:46.839300+0000","last_undegraded":"2024-05-11T01:14:46.839300+0000","last_fullsized":"2024-05-11T01:14:46.839300+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854582+0000","last_change":"2024-05-11T01:13:59.431401+0000","last_active":"2024-05-11T01:14:46.854582+0000","last_peered":"2024-05-11T01:14:46.854582+0000","last_clean":"2024-05-11T01:14:46.854582+0000","last_became_active":"2024-05-11T01:13:59.431126+0000","last_became_peered":"2024-05-11T01:13:59.431126+0000","last_unstale":"2024-05-11T01:14:46.854582+0000","last_undegraded":"2024-05-11T01:14:46.854582+0000","last_fullsized":"2024-05-11T01:14:46.854582+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:42:47.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855202+0000","last_change":"2024-05-11T01:13:58.431417+0000","last_active":"2024-05-11T01:14:46.855202+0000","last_peered":"2024-05-11T01:14:46.855202+0000","last_clean":"2024-05-11T01:14:46.855202+0000","last_became_active":"2024-05-11T01:13:58.431132+0000","last_became_peered":"2024-05-11T01:13:58.431132+0000","last_unstale":"2024-05-11T01:14:46.855202+0000","last_undegraded":"2024-05-11T01:14:46.855202+0000","last_fullsized":"2024-05-11T01:14:46.855202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:23:16.977274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836973+0000","last_change":"2024-05-11T01:13:59.436396+0000","last_active":"2024-05-11T01:14:46.836973+0000","last_peered":"2024-05-11T01:14:46.836973+0000","last_clean":"2024-05-11T01:14:46.836973+0000","last_became_active":"2024-05-11T01:13:59.436043+0000","last_became_peered":"2024-05-11T01:13:59.436043+0000","last_unstale":"2024-05-11T01:14:46.836973+0000","last_undegraded":"2024-05-11T01:14:46.836973+0000","last_fullsized":"2024-05-11T01:14:46.836973+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:25:32.761359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854476+0000","last_change":"2024-05-11T01:13:58.435566+0000","last_active":"2024-05-11T01:14:46.854476+0000","last_peered":"2024-05-11T01:14:46.854476+0000","last_clean":"2024-05-11T01:14:46.854476+0000","last_became_active":"2024-05-11T01:13:58.435280+0000","last_became_peered":"2024-05-11T01:13:58.435280+0000","last_unstale":"2024-05-11T01:14:46.854476+0000","last_undegraded":"2024-05-11T01:14:46.854476+0000","last_fullsized":"2024-05-11T01:14:46.854476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:09:47.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854368+0000","last_change":"2024-05-11T01:13:59.432307+0000","last_active":"2024-05-11T01:14:46.854368+0000","last_peered":"2024-05-11T01:14:46.854368+0000","last_clean":"2024-05-11T01:14:46.854368+0000","last_became_active":"2024-05-11T01:13:59.431913+0000","last_became_peered":"2024-05-11T01:13:59.431913+0000","last_unstale":"2024-05-11T01:14:46.854368+0000","last_undegraded":"2024-05-11T01:14:46.854368+0000","last_fullsized":"2024-05-11T01:14:46.854368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"63'10","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:07.973661+0000","last_change":"2024-05-11T01:13:58.425157+0000","last_active":"2024-05-11T01:16:07.973661+0000","last_peered":"2024-05-11T01:16:07.973661+0000","last_clean":"2024-05-11T01:16:07.973661+0000","last_became_active":"2024-05-11T01:13:58.424894+0000","last_became_peered":"2024-05-11T01:13:58.424894+0000","last_unstale":"2024-05-11T01:16:07.973661+0000","last_undegraded":"2024-05-11T01:16:07.973661+0000","last_fullsized":"2024-05-11T01:16:07.973661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:48:11.424880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837159+0000","last_change":"2024-05-11T01:13:59.447362+0000","last_active":"2024-05-11T01:14:46.837159+0000","last_peered":"2024-05-11T01:14:46.837159+0000","last_clean":"2024-05-11T01:14:46.837159+0000","last_became_active":"2024-05-11T01:13:59.447082+0000","last_became_peered":"2024-05-11T01:13:59.447082+0000","last_unstale":"2024-05-11T01:14:46.837159+0000","last_undegraded":"2024-05-11T01:14:46.837159+0000","last_fullsized":"2024-05-11T01:14:46.837159+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:12:13.972035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837499+0000","last_change":"2024-05-11T01:13:58.439044+0000","last_active":"2024-05-11T01:14:46.837499+0000","last_peered":"2024-05-11T01:14:46.837499+0000","last_clean":"2024-05-11T01:14:46.837499+0000","last_became_active":"2024-05-11T01:13:58.438603+0000","last_became_peered":"2024-05-11T01:13:58.438603+0000","last_unstale":"2024-05-11T01:14:46.837499+0000","last_undegraded":"2024-05-11T01:14:46.837499+0000","last_fullsized":"2024-05-11T01:14:46.837499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854651+0000","last_change":"2024-05-11T01:13:59.449114+0000","last_active":"2024-05-11T01:14:46.854651+0000","last_peered":"2024-05-11T01:14:46.854651+0000","last_clean":"2024-05-11T01:14:46.854651+0000","last_became_active":"2024-05-11T01:13:59.448532+0000","last_became_peered":"2024-05-11T01:13:59.448532+0000","last_unstale":"2024-05-11T01:14:46.854651+0000","last_undegraded":"2024-05-11T01:14:46.854651+0000","last_fullsized":"2024-05-11T01:14:46.854651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:59:07.940383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854151+0000","last_change":"2024-05-11T01:13:58.434118+0000","last_active":"2024-05-11T01:14:46.854151+0000","last_peered":"2024-05-11T01:14:46.854151+0000","last_clean":"2024-05-11T01:14:46.854151+0000","last_became_active":"2024-05-11T01:13:58.433818+0000","last_became_peered":"2024-05-11T01:13:58.433818+0000","last_unstale":"2024-05-11T01:14:46.854151+0000","last_undegraded":"2024-05-11T01:14:46.854151+0000","last_fullsized":"2024-05-11T01:14:46.854151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:19:49.867304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836702+0000","last_change":"2024-05-11T01:13:59.435013+0000","last_active":"2024-05-11T01:14:46.836702+0000","last_peered":"2024-05-11T01:14:46.836702+0000","last_clean":"2024-05-11T01:14:46.836702+0000","last_became_active":"2024-05-11T01:13:59.434726+0000","last_became_peered":"2024-05-11T01:13:59.434726+0000","last_unstale":"2024-05-11T01:14:46.836702+0000","last_undegraded":"2024-05-11T01:14:46.836702+0000","last_fullsized":"2024-05-11T01:14:46.836702+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:03:22.763694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837125+0000","last_change":"2024-05-11T01:13:58.438894+0000","last_active":"2024-05-11T01:14:46.837125+0000","last_peered":"2024-05-11T01:14:46.837125+0000","last_clean":"2024-05-11T01:14:46.837125+0000","last_became_active":"2024-05-11T01:13:58.438528+0000","last_became_peered":"2024-05-11T01:13:58.438528+0000","last_unstale":"2024-05-11T01:14:46.837125+0000","last_undegraded":"2024-05-11T01:14:46.837125+0000","last_fullsized":"2024-05-11T01:14:46.837125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:14:17.884412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854117+0000","last_change":"2024-05-11T01:13:59.437777+0000","last_active":"2024-05-11T01:14:46.854117+0000","last_peered":"2024-05-11T01:14:46.854117+0000","last_clean":"2024-05-11T01:14:46.854117+0000","last_became_active":"2024-05-11T01:13:59.437176+0000","last_became_peered":"2024-05-11T01:13:59.437176+0000","last_unstale":"2024-05-11T01:14:46.854117+0000","last_undegraded":"2024-05-11T01:14:46.854117+0000","last_fullsized":"2024-05-11T01:14:46.854117+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:07:21.319542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854062+0000","last_change":"2024-05-11T01:13:58.431218+0000","last_active":"2024-05-11T01:14:46.854062+0000","last_peered":"2024-05-11T01:14:46.854062+0000","last_clean":"2024-05-11T01:14:46.854062+0000","last_became_active":"2024-05-11T01:13:58.430952+0000","last_became_peered":"2024-05-11T01:13:58.430952+0000","last_unstale":"2024-05-11T01:14:46.854062+0000","last_undegraded":"2024-05-11T01:14:46.854062+0000","last_fullsized":"2024-05-11T01:14:46.854062+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:03:39.335329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854717+0000","last_change":"2024-05-11T01:13:59.427029+0000","last_active":"2024-05-11T01:14:46.854717+0000","last_peered":"2024-05-11T01:14:46.854717+0000","last_clean":"2024-05-11T01:14:46.854717+0000","last_became_active":"2024-05-11T01:13:59.426727+0000","last_became_peered":"2024-05-11T01:13:59.426727+0000","last_unstale":"2024-05-11T01:14:46.854717+0000","last_undegraded":"2024-05-11T01:14:46.854717+0000","last_fullsized":"2024-05-11T01:14:46.854717+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:52:52.626350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854265+0000","last_change":"2024-05-11T01:13:58.434507+0000","last_active":"2024-05-11T01:14:46.854265+0000","last_peered":"2024-05-11T01:14:46.854265+0000","last_clean":"2024-05-11T01:14:46.854265+0000","last_became_active":"2024-05-11T01:13:58.434215+0000","last_became_peered":"2024-05-11T01:13:58.434215+0000","last_unstale":"2024-05-11T01:14:46.854265+0000","last_undegraded":"2024-05-11T01:14:46.854265+0000","last_fullsized":"2024-05-11T01:14:46.854265+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:07:35.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854830+0000","last_change":"2024-05-11T01:13:59.444388+0000","last_active":"2024-05-11T01:14:46.854830+0000","last_peered":"2024-05-11T01:14:46.854830+0000","last_clean":"2024-05-11T01:14:46.854830+0000","last_became_active":"2024-05-11T01:13:59.443981+0000","last_became_peered":"2024-05-11T01:13:59.443981+0000","last_unstale":"2024-05-11T01:14:46.854830+0000","last_undegraded":"2024-05-11T01:14:46.854830+0000","last_fullsized":"2024-05-11T01:14:46.854830+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:30:26.909526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839352+0000","last_change":"2024-05-11T01:13:58.429801+0000","last_active":"2024-05-11T01:14:46.839352+0000","last_peered":"2024-05-11T01:14:46.839352+0000","last_clean":"2024-05-11T01:14:46.839352+0000","last_became_active":"2024-05-11T01:13:58.429538+0000","last_became_peered":"2024-05-11T01:13:58.429538+0000","last_unstale":"2024-05-11T01:14:46.839352+0000","last_undegraded":"2024-05-11T01:14:46.839352+0000","last_fullsized":"2024-05-11T01:14:46.839352+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:48:27.314762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837386+0000","last_change":"2024-05-11T01:13:59.438763+0000","last_active":"2024-05-11T01:14:46.837386+0000","last_peered":"2024-05-11T01:14:46.837386+0000","last_clean":"2024-05-11T01:14:46.837386+0000","last_became_active":"2024-05-11T01:13:59.438474+0000","last_became_peered":"2024-05-11T01:13:59.438474+0000","last_unstale":"2024-05-11T01:14:46.837386+0000","last_undegraded":"2024-05-11T01:14:46.837386+0000","last_fullsized":"2024-05-11T01:14:46.837386+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:51:33.643462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854341+0000","last_change":"2024-05-11T01:13:58.426925+0000","last_active":"2024-05-11T01:14:46.854341+0000","last_peered":"2024-05-11T01:14:46.854341+0000","last_clean":"2024-05-11T01:14:46.854341+0000","last_became_active":"2024-05-11T01:13:58.426586+0000","last_became_peered":"2024-05-11T01:13:58.426586+0000","last_unstale":"2024-05-11T01:14:46.854341+0000","last_undegraded":"2024-05-11T01:14:46.854341+0000","last_fullsized":"2024-05-11T01:14:46.854341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:48:36.869416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837516+0000","last_change":"2024-05-11T01:13:59.436970+0000","last_active":"2024-05-11T01:14:46.837516+0000","last_peered":"2024-05-11T01:14:46.837516+0000","last_clean":"2024-05-11T01:14:46.837516+0000","last_became_active":"2024-05-11T01:13:59.436373+0000","last_became_peered":"2024-05-11T01:13:59.436373+0000","last_unstale":"2024-05-11T01:14:46.837516+0000","last_undegraded":"2024-05-11T01:14:46.837516+0000","last_fullsized":"2024-05-11T01:14:46.837516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:29:26.197889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837121+0000","last_change":"2024-05-11T01:13:58.439680+0000","last_active":"2024-05-11T01:14:46.837121+0000","last_peered":"2024-05-11T01:14:46.837121+0000","last_clean":"2024-05-11T01:14:46.837121+0000","last_became_active":"2024-05-11T01:13:58.439404+0000","last_became_peered":"2024-05-11T01:13:58.439404+0000","last_unstale":"2024-05-11T01:14:46.837121+0000","last_undegraded":"2024-05-11T01:14:46.837121+0000","last_fullsized":"2024-05-11T01:14:46.837121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:41:26.236385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837268+0000","last_change":"2024-05-11T01:13:59.435753+0000","last_active":"2024-05-11T01:14:46.837268+0000","last_peered":"2024-05-11T01:14:46.837268+0000","last_clean":"2024-05-11T01:14:46.837268+0000","last_became_active":"2024-05-11T01:13:59.435253+0000","last_became_peered":"2024-05-11T01:13:59.435253+0000","last_unstale":"2024-05-11T01:14:46.837268+0000","last_undegraded":"2024-05-11T01:14:46.837268+0000","last_fullsized":"2024-05-11T01:14:46.837268+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:06:58.758258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837621+0000","last_change":"2024-05-11T01:13:58.437862+0000","last_active":"2024-05-11T01:14:46.837621+0000","last_peered":"2024-05-11T01:14:46.837621+0000","last_clean":"2024-05-11T01:14:46.837621+0000","last_became_active":"2024-05-11T01:13:58.437597+0000","last_became_peered":"2024-05-11T01:13:58.437597+0000","last_unstale":"2024-05-11T01:14:46.837621+0000","last_undegraded":"2024-05-11T01:14:46.837621+0000","last_fullsized":"2024-05-11T01:14:46.837621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:19:23.818943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836499+0000","last_change":"2024-05-11T01:13:59.431673+0000","last_active":"2024-05-11T01:14:46.836499+0000","last_peered":"2024-05-11T01:14:46.836499+0000","last_clean":"2024-05-11T01:14:46.836499+0000","last_became_active":"2024-05-11T01:13:59.431386+0000","last_became_peered":"2024-05-11T01:13:59.431386+0000","last_unstale":"2024-05-11T01:14:46.836499+0000","last_undegraded":"2024-05-11T01:14:46.836499+0000","last_fullsized":"2024-05-11T01:14:46.836499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:27.086216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853615+0000","last_change":"2024-05-11T01:13:58.431189+0000","last_active":"2024-05-11T01:14:46.853615+0000","last_peered":"2024-05-11T01:14:46.853615+0000","last_clean":"2024-05-11T01:14:46.853615+0000","last_became_active":"2024-05-11T01:13:58.430890+0000","last_became_peered":"2024-05-11T01:13:58.430890+0000","last_unstale":"2024-05-11T01:14:46.853615+0000","last_undegraded":"2024-05-11T01:14:46.853615+0000","last_fullsized":"2024-05-11T01:14:46.853615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:55.053612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854490+0000","last_change":"2024-05-11T01:13:59.429997+0000","last_active":"2024-05-11T01:14:46.854490+0000","last_peered":"2024-05-11T01:14:46.854490+0000","last_clean":"2024-05-11T01:14:46.854490+0000","last_became_active":"2024-05-11T01:13:59.429734+0000","last_became_peered":"2024-05-11T01:13:59.429734+0000","last_unstale":"2024-05-11T01:14:46.854490+0000","last_undegraded":"2024-05-11T01:14:46.854490+0000","last_fullsized":"2024-05-11T01:14:46.854490+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:53:47.445191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854269+0000","last_change":"2024-05-11T01:13:58.431136+0000","last_active":"2024-05-11T01:14:46.854269+0000","last_peered":"2024-05-11T01:14:46.854269+0000","last_clean":"2024-05-11T01:14:46.854269+0000","last_became_active":"2024-05-11T01:13:58.430846+0000","last_became_peered":"2024-05-11T01:13:58.430846+0000","last_unstale":"2024-05-11T01:14:46.854269+0000","last_undegraded":"2024-05-11T01:14:46.854269+0000","last_fullsized":"2024-05-11T01:14:46.854269+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:38.439492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854167+0000","last_change":"2024-05-11T01:13:59.450616+0000","last_active":"2024-05-11T01:14:46.854167+0000","last_peered":"2024-05-11T01:14:46.854167+0000","last_clean":"2024-05-11T01:14:46.854167+0000","last_became_active":"2024-05-11T01:13:59.450300+0000","last_became_peered":"2024-05-11T01:13:59.450300+0000","last_unstale":"2024-05-11T01:14:46.854167+0000","last_undegraded":"2024-05-11T01:14:46.854167+0000","last_fullsized":"2024-05-11T01:14:46.854167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:37:45.777924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854250+0000","last_change":"2024-05-11T01:13:58.436187+0000","last_active":"2024-05-11T01:14:46.854250+0000","last_peered":"2024-05-11T01:14:46.854250+0000","last_clean":"2024-05-11T01:14:46.854250+0000","last_became_active":"2024-05-11T01:13:58.434702+0000","last_became_peered":"2024-05-11T01:13:58.434702+0000","last_unstale":"2024-05-11T01:14:46.854250+0000","last_undegraded":"2024-05-11T01:14:46.854250+0000","last_fullsized":"2024-05-11T01:14:46.854250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:40:28.737362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837639+0000","last_change":"2024-05-11T01:13:59.452028+0000","last_active":"2024-05-11T01:14:46.837639+0000","last_peered":"2024-05-11T01:14:46.837639+0000","last_clean":"2024-05-11T01:14:46.837639+0000","last_became_active":"2024-05-11T01:13:59.451773+0000","last_became_peered":"2024-05-11T01:13:59.451773+0000","last_unstale":"2024-05-11T01:14:46.837639+0000","last_undegraded":"2024-05-11T01:14:46.837639+0000","last_fullsized":"2024-05-11T01:14:46.837639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:54:02.542792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"61'1","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837109+0000","last_change":"2024-05-11T01:13:58.436921+0000","last_active":"2024-05-11T01:14:46.837109+0000","last_peered":"2024-05-11T01:14:46.837109+0000","last_clean":"2024-05-11T01:14:46.837109+0000","last_became_active":"2024-05-11T01:13:58.436650+0000","last_became_peered":"2024-05-11T01:13:58.436650+0000","last_unstale":"2024-05-11T01:14:46.837109+0000","last_undegraded":"2024-05-11T01:14:46.837109+0000","last_fullsized":"2024-05-11T01:14:46.837109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:16:06.268571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854569+0000","last_change":"2024-05-11T01:13:59.442952+0000","last_active":"2024-05-11T01:14:46.854569+0000","last_peered":"2024-05-11T01:14:46.854569+0000","last_clean":"2024-05-11T01:14:46.854569+0000","last_became_active":"2024-05-11T01:13:59.442413+0000","last_became_peered":"2024-05-11T01:13:59.442413+0000","last_unstale":"2024-05-11T01:14:46.854569+0000","last_undegraded":"2024-05-11T01:14:46.854569+0000","last_fullsized":"2024-05-11T01:14:46.854569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:32:54.808774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854522+0000","last_change":"2024-05-11T01:13:58.436342+0000","last_active":"2024-05-11T01:14:46.854522+0000","last_peered":"2024-05-11T01:14:46.854522+0000","last_clean":"2024-05-11T01:14:46.854522+0000","last_became_active":"2024-05-11T01:13:58.436037+0000","last_became_peered":"2024-05-11T01:13:58.436037+0000","last_unstale":"2024-05-11T01:14:46.854522+0000","last_undegraded":"2024-05-11T01:14:46.854522+0000","last_fullsized":"2024-05-11T01:14:46.854522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:22.498662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854431+0000","last_change":"2024-05-11T01:13:59.434239+0000","last_active":"2024-05-11T01:14:46.854431+0000","last_peered":"2024-05-11T01:14:46.854431+0000","last_clean":"2024-05-11T01:14:46.854431+0000","last_became_active":"2024-05-11T01:13:59.433946+0000","last_became_peered":"2024-05-11T01:13:59.433946+0000","last_unstale":"2024-05-11T01:14:46.854431+0000","last_undegraded":"2024-05-11T01:14:46.854431+0000","last_fullsized":"2024-05-11T01:14:46.854431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:07:36.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837338+0000","last_change":"2024-05-11T01:13:58.432458+0000","last_active":"2024-05-11T01:14:46.837338+0000","last_peered":"2024-05-11T01:14:46.837338+0000","last_clean":"2024-05-11T01:14:46.837338+0000","last_became_active":"2024-05-11T01:13:58.432149+0000","last_became_peered":"2024-05-11T01:13:58.432149+0000","last_unstale":"2024-05-11T01:14:46.837338+0000","last_undegraded":"2024-05-11T01:14:46.837338+0000","last_fullsized":"2024-05-11T01:14:46.837338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837389+0000","last_change":"2024-05-11T01:13:59.432345+0000","last_active":"2024-05-11T01:14:46.837389+0000","last_peered":"2024-05-11T01:14:46.837389+0000","last_clean":"2024-05-11T01:14:46.837389+0000","last_became_active":"2024-05-11T01:13:59.432058+0000","last_became_peered":"2024-05-11T01:13:59.432058+0000","last_unstale":"2024-05-11T01:14:46.837389+0000","last_undegraded":"2024-05-11T01:14:46.837389+0000","last_fullsized":"2024-05-11T01:14:46.837389+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:38:32.025100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854602+0000","last_change":"2024-05-11T01:13:58.433877+0000","last_active":"2024-05-11T01:14:46.854602+0000","last_peered":"2024-05-11T01:14:46.854602+0000","last_clean":"2024-05-11T01:14:46.854602+0000","last_became_active":"2024-05-11T01:13:58.433584+0000","last_became_peered":"2024-05-11T01:13:58.433584+0000","last_unstale":"2024-05-11T01:14:46.854602+0000","last_undegraded":"2024-05-11T01:14:46.854602+0000","last_fullsized":"2024-05-11T01:14:46.854602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:20.805314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.837457+0000","last_change":"2024-05-11T01:13:59.434011+0000","last_active":"2024-05-11T01:14:46.837457+0000","last_peered":"2024-05-11T01:14:46.837457+0000","last_clean":"2024-05-11T01:14:46.837457+0000","last_became_active":"2024-05-11T01:13:59.432337+0000","last_became_peered":"2024-05-11T01:13:59.432337+0000","last_unstale":"2024-05-11T01:14:46.837457+0000","last_undegraded":"2024-05-11T01:14:46.837457+0000","last_fullsized":"2024-05-11T01:14:46.837457+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:21:41.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.839100+0000","last_change":"2024-05-11T01:13:58.435211+0000","last_active":"2024-05-11T01:14:46.839100+0000","last_peered":"2024-05-11T01:14:46.839100+0000","last_clean":"2024-05-11T01:14:46.839100+0000","last_became_active":"2024-05-11T01:13:58.434956+0000","last_became_peered":"2024-05-11T01:13:58.434956+0000","last_unstale":"2024-05-11T01:14:46.839100+0000","last_undegraded":"2024-05-11T01:14:46.839100+0000","last_fullsized":"2024-05-11T01:14:46.839100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:00:00.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853560+0000","last_change":"2024-05-11T01:13:59.432319+0000","last_active":"2024-05-11T01:14:46.853560+0000","last_peered":"2024-05-11T01:14:46.853560+0000","last_clean":"2024-05-11T01:14:46.853560+0000","last_became_active":"2024-05-11T01:13:59.432012+0000","last_became_peered":"2024-05-11T01:13:59.432012+0000","last_unstale":"2024-05-11T01:14:46.853560+0000","last_undegraded":"2024-05-11T01:14:46.853560+0000","last_fullsized":"2024-05-11T01:14:46.853560+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:49:21.807568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.836595+0000","last_change":"2024-05-11T01:13:58.433752+0000","last_active":"2024-05-11T01:14:46.836595+0000","last_peered":"2024-05-11T01:14:46.836595+0000","last_clean":"2024-05-11T01:14:46.836595+0000","last_became_active":"2024-05-11T01:13:58.433411+0000","last_became_peered":"2024-05-11T01:13:58.433411+0000","last_unstale":"2024-05-11T01:14:46.836595+0000","last_undegraded":"2024-05-11T01:14:46.836595+0000","last_fullsized":"2024-05-11T01:14:46.836595+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:54:40.379141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.838906+0000","last_change":"2024-05-11T01:13:59.446971+0000","last_active":"2024-05-11T01:14:46.838906+0000","last_peered":"2024-05-11T01:14:46.838906+0000","last_clean":"2024-05-11T01:14:46.838906+0000","last_became_active":"2024-05-11T01:13:59.446670+0000","last_became_peered":"2024-05-11T01:13:59.446670+0000","last_unstale":"2024-05-11T01:14:46.838906+0000","last_undegraded":"2024-05-11T01:14:46.838906+0000","last_fullsized":"2024-05-11T01:14:46.838906+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:45:46.843918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854241+0000","last_change":"2024-05-11T01:13:58.435876+0000","last_active":"2024-05-11T01:14:46.854241+0000","last_peered":"2024-05-11T01:14:46.854241+0000","last_clean":"2024-05-11T01:14:46.854241+0000","last_became_active":"2024-05-11T01:13:58.434062+0000","last_became_peered":"2024-05-11T01:13:58.434062+0000","last_unstale":"2024-05-11T01:14:46.854241+0000","last_undegraded":"2024-05-11T01:14:46.854241+0000","last_fullsized":"2024-05-11T01:14:46.854241+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:12:26.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.855069+0000","last_change":"2024-05-11T01:13:59.445426+0000","last_active":"2024-05-11T01:14:46.855069+0000","last_peered":"2024-05-11T01:14:46.855069+0000","last_clean":"2024-05-11T01:14:46.855069+0000","last_became_active":"2024-05-11T01:13:59.445146+0000","last_became_peered":"2024-05-11T01:13:59.445146+0000","last_unstale":"2024-05-11T01:14:46.855069+0000","last_undegraded":"2024-05-11T01:14:46.855069+0000","last_fullsized":"2024-05-11T01:14:46.855069+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:58.410869+0000","last_clean_scrub_stamp":"2024-05-11T01:13:58.410869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:57:36.879340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":21,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.854223+0000","last_change":"2024-05-11T01:13:58.434600+0000","last_active":"2024-05-11T01:14:46.854223+0000","last_peered":"2024-05-11T01:14:46.854223+0000","last_clean":"2024-05-11T01:14:46.854223+0000","last_became_active":"2024-05-11T01:13:58.434320+0000","last_became_peered":"2024-05-11T01:13:58.434320+0000","last_unstale":"2024-05-11T01:14:46.854223+0000","last_undegraded":"2024-05-11T01:14:46.854223+0000","last_fullsized":"2024-05-11T01:14:46.854223+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:13:57.408615+0000","last_clean_scrub_stamp":"2024-05-11T01:13:57.408615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:57.275676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"51'86","reported_seq":246,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:14:46.853980+0000","last_change":"2024-05-11T01:00:57.065676+0000","last_active":"2024-05-11T01:14:46.853980+0000","last_peered":"2024-05-11T01:14:46.853980+0000","last_clean":"2024-05-11T01:14:46.853980+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:14:46.853980+0000","last_undegraded":"2024-05-11T01:14:46.853980+0000","last_fullsized":"2024-05-11T01:14:46.853980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:00:55.632733+0000","last_clean_scrub_stamp":"2024-05-11T01:00:55.632733+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:38:50.470482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3807,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575034,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":949409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542332,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":757760,"data_stored":359623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575034,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575034,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54128,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317712,"statfs":{"total":96636764160,"available":96581337088,"internally_reserved":0,"allocated":761856,"data_stored":361960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575034,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44408,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327432,"statfs":{"total":96636764160,"available":96591290368,"internally_reserved":0,"allocated":761856,"data_stored":359589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575035,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":765952,"data_stored":361982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58288,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313552,"statfs":{"total":96636764160,"available":96577077248,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575034,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1347584,"data_stored":949353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:16:21.349 INFO:tasks.ceph:pgid 7.3f last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.349 INFO:tasks.ceph:pgid 7.3e last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.349 INFO:tasks.ceph:pgid 7.3d last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.349 INFO:tasks.ceph:pgid 7.3c last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.349 INFO:tasks.ceph:pgid 7.3b last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.349 INFO:tasks.ceph:pgid 7.3a last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.350 INFO:tasks.ceph:pgid 7.39 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.350 INFO:tasks.ceph:pgid 7.38 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.350 INFO:tasks.ceph:pgid 7.37 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.350 INFO:tasks.ceph:pgid 7.36 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.350 INFO:tasks.ceph:pgid 7.35 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.350 INFO:tasks.ceph:pgid 7.34 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.351 INFO:tasks.ceph:pgid 7.33 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.351 INFO:tasks.ceph:pgid 7.32 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.351 INFO:tasks.ceph:pgid 7.31 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.351 INFO:tasks.ceph:pgid 7.30 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.351 INFO:tasks.ceph:pgid 7.2f last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.351 INFO:tasks.ceph:pgid 7.2e last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.352 INFO:tasks.ceph:pgid 7.2d last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.352 INFO:tasks.ceph:pgid 7.2c last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.352 INFO:tasks.ceph:pgid 7.2b last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.352 INFO:tasks.ceph:pgid 7.2a last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.352 INFO:tasks.ceph:pgid 7.29 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.353 INFO:tasks.ceph:pgid 7.28 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.353 INFO:tasks.ceph:pgid 7.27 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.353 INFO:tasks.ceph:pgid 7.26 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.353 INFO:tasks.ceph:pgid 7.25 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.354 INFO:tasks.ceph:pgid 7.24 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.354 INFO:tasks.ceph:pgid 7.23 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.354 INFO:tasks.ceph:pgid 7.22 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.354 INFO:tasks.ceph:pgid 7.21 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.354 INFO:tasks.ceph:pgid 7.20 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.354 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.355 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.355 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.355 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.355 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.355 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.355 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.356 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.356 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.356 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.356 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.356 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.357 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.357 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.357 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.357 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.357 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.357 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.358 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.358 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.358 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.358 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.358 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.358 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.359 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.359 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.359 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.359 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.359 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.359 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.360 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.360 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.360 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.360 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.360 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.360 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.361 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.361 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.361 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.361 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.361 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.361 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.362 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.362 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.362 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.362 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.362 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.362 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.363 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.363 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.363 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.363 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.363 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.363 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.364 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.364 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.364 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.364 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.364 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.364 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.365 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.365 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.365 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-05-11T01:13:58.410869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=58, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.365 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-05-11T01:13:57.408615+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=57, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.365 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-11T01:00:55.632733+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=0, tm_sec=55, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=1, tm_min=16, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T01:16:21.365 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-11T01:16:41.366 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T01:16:41.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.483+0000 7f877e76a640 1 Processor -- start 2024-05-11T01:16:41.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.483+0000 7f877e76a640 1 -- start start 2024-05-11T01:16:41.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.484+0000 7f877e76a640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8778072ce0 0x7f877806e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.484+0000 7f877e76a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87780afb30 0x7f87780730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.484+0000 7f877e76a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87780aea90 con 0x7f8778073600 2024-05-11T01:16:41.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.484+0000 7f877e76a640 1 -- --> v2:172.21.15.52:3300/0 -- mon_getmap magic: 0 -- 0x7f87780b59e0 con 0x7f8778072ce0 2024-05-11T01:16:41.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.484+0000 7f877e76a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f87780ae860 con 0x7f87780afb30 2024-05-11T01:16:41.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.484+0000 7f8777fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8778072ce0 0x7f877806e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.484+0000 7f8777fff640 1 --2- >> v2:172.21.15.52:3300/0 conn(0x7f8778072ce0 0x7f877806e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52296/0 (socket says 172.21.15.52:52296) 2024-05-11T01:16:41.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.484+0000 7f8777fff640 1 -- 172.21.15.52:0/888468043 learned_addr learned my addr 172.21.15.52:0/888468043 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:41.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.484+0000 7f87777fe640 1 --2- 172.21.15.52:0/888468043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87780afb30 0x7f87780730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.485+0000 7f8777fff640 1 -- 172.21.15.52:0/888468043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87780afb30 msgr2=0x7f87780730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:41.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.485+0000 7f8777fff640 1 --2- 172.21.15.52:0/888468043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87780afb30 0x7f87780730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.485+0000 7f8777fff640 1 -- 172.21.15.52:0/888468043 >> v1:172.21.15.119:6789/0 conn(0x7f8778073600 legacy=0x7f87780b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.485+0000 7f8777fff640 1 -- 172.21.15.52:0/888468043 --> v2:172.21.15.52:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87780aed70 con 0x7f8778072ce0 2024-05-11T01:16:41.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.485+0000 7f8777fff640 1 --2- 172.21.15.52:0/888468043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8778072ce0 0x7f877806e050 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8768009870 tx=0x7f876802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afd71cd7e7f6b3d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:41.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f8776ffd640 1 -- 172.21.15.52:0/888468043 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768037070 con 0x7f8778072ce0 2024-05-11T01:16:41.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f8776ffd640 1 -- 172.21.15.52:0/888468043 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768032440 con 0x7f8778072ce0 2024-05-11T01:16:41.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f8776ffd640 1 -- 172.21.15.52:0/888468043 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768032810 con 0x7f8778072ce0 2024-05-11T01:16:41.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f877e76a640 1 -- 172.21.15.52:0/888468043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8778072ce0 msgr2=0x7f877806e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f877e76a640 1 --2- 172.21.15.52:0/888468043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8778072ce0 0x7f877806e050 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8768009870 tx=0x7f876802a270 comp rx=0 tx=0).stop 2024-05-11T01:16:41.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f877e76a640 1 -- 172.21.15.52:0/888468043 shutdown_connections 2024-05-11T01:16:41.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f877e76a640 1 --2- 172.21.15.52:0/888468043 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87780afb30 0x7f87780730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f877e76a640 1 --2- 172.21.15.52:0/888468043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8778072ce0 0x7f877806e050 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f877e76a640 1 -- 172.21.15.52:0/888468043 >> 172.21.15.52:0/888468043 conn(0x7f87780b2a00 msgr2=0x7f87780b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:41.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f877e76a640 1 -- 172.21.15.52:0/888468043 shutdown_connections 2024-05-11T01:16:41.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.486+0000 7f877e76a640 1 -- 172.21.15.52:0/888468043 wait complete. 2024-05-11T01:16:41.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.487+0000 7f877e76a640 1 Processor -- start 2024-05-11T01:16:41.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.488+0000 7f877e76a640 1 -- start start 2024-05-11T01:16:41.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.488+0000 7f877e76a640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8778072ce0 0x7f8778148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.488+0000 7f877e76a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8778073600 0x7f877815e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.488+0000 7f877e76a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87780afb30 0x7f877809ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.488+0000 7f877e76a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87780af610 con 0x7f87780afb30 2024-05-11T01:16:41.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.488+0000 7f877e76a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f87780af330 con 0x7f8778073600 2024-05-11T01:16:41.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.488+0000 7f877e76a640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7f8778073a50 con 0x7f8778072ce0 2024-05-11T01:16:41.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.488+0000 7f877cd07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87780afb30 0x7f877809ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877cd07640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87780afb30 0x7f877809ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52310/0 (socket says 172.21.15.52:52310) 2024-05-11T01:16:41.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877cd07640 1 -- 172.21.15.52:0/1195989064 learned_addr learned my addr 172.21.15.52:0/1195989064 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:41.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f8777fff640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8778072ce0 0x7f8778148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f87777fe640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8778073600 0x7f877815e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877cd07640 1 -- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8778072ce0 msgr2=0x7f8778148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877cd07640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8778072ce0 0x7f8778148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877cd07640 1 -- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8778073600 msgr2=0x7f877815e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877cd07640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8778073600 0x7f877815e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877cd07640 1 -- 172.21.15.52:0/1195989064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877815f0c0 con 0x7f87780afb30 2024-05-11T01:16:41.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f87777fe640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8778073600 0x7f877815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T01:16:41.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877cd07640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87780afb30 0x7f877809ca90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f877000c7d0 tx=0x7f877000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:41.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f8774ff9640 1 -- 172.21.15.52:0/1195989064 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770003b70 con 0x7f87780afb30 2024-05-11T01:16:41.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f8774ff9640 1 -- 172.21.15.52:0/1195989064 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8770003d10 con 0x7f87780afb30 2024-05-11T01:16:41.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877809bcd0 con 0x7f87780afb30 2024-05-11T01:16:41.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f8774ff9640 1 -- 172.21.15.52:0/1195989064 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877000aa70 con 0x7f87780afb30 2024-05-11T01:16:41.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.489+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877809c290 con 0x7f87780afb30 2024-05-11T01:16:41.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.490+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8778069780 con 0x7f87780afb30 2024-05-11T01:16:41.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.491+0000 7f8774ff9640 1 -- 172.21.15.52:0/1195989064 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7f877000ac10 con 0x7f87780afb30 2024-05-11T01:16:41.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.491+0000 7f8774ff9640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87640762d0 0x7f8764078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.492+0000 7f8774ff9640 1 -- 172.21.15.52:0/1195989064 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7f8770097740 con 0x7f87780afb30 2024-05-11T01:16:41.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.492+0000 7f8777fff640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87640762d0 0x7f8764078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.492+0000 7f8777fff640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87640762d0 0x7f8764078790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8768034000 tx=0x7f876802e000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:41.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.496+0000 7f8774ff9640 1 -- 172.21.15.52:0/1195989064 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8770060ec0 con 0x7f87780afb30 2024-05-11T01:16:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.682+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 --> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87780b2540 con 0x7f87640762d0 2024-05-11T01:16:41.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.689+0000 7f8774ff9640 1 -- 172.21.15.52:0/1195989064 <== mgr.5260 v2:172.21.15.119:6832/4040438696 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251894 (secure 0 0 0) 0x7f87780b2540 con 0x7f87640762d0 2024-05-11T01:16:41.690 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:16:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-05-11T01:16:41.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87640762d0 msgr2=0x7f8764078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87640762d0 0x7f8764078790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8768034000 tx=0x7f876802e000 comp rx=0 tx=0).stop 2024-05-11T01:16:41.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87780afb30 msgr2=0x7f877809ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87780afb30 0x7f877809ca90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f877000c7d0 tx=0x7f877000cc90 comp rx=0 tx=0).stop 2024-05-11T01:16:41.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 shutdown_connections 2024-05-11T01:16:41.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7f87640762d0 0x7f8764078790 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87780afb30 0x7f877809ca90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8778073600 0x7f877815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 --2- 172.21.15.52:0/1195989064 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8778072ce0 0x7f8778148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 >> 172.21.15.52:0/1195989064 conn(0x7f87780b2a00 msgr2=0x7f8778152380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:41.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 shutdown_connections 2024-05-11T01:16:41.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.692+0000 7f877e76a640 1 -- 172.21.15.52:0/1195989064 wait complete. 2024-05-11T01:16:41.702 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-05-11T01:16:39.746331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594175,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":174,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300904,"kb_used_data":7080,"kb_used_omap":12,"kb_used_meta":293747,"kb_avail":754673816,"statfs":{"total":773094113280,"available":772785987584,"internally_reserved":0,"allocated":7249920,"data_stored":4059958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":300797489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002876"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:32.111165+0000","last_change":"2024-05-11T01:16:32.111208+0000","last_active":"2024-05-11T01:16:32.111165+0000","last_peered":"2024-05-11T01:16:32.111165+0000","last_clean":"2024-05-11T01:16:32.111165+0000","last_became_active":"2024-05-11T01:13:59.434594+0000","last_became_peered":"2024-05-11T01:13:59.434594+0000","last_unstale":"2024-05-11T01:16:32.111165+0000","last_undegraded":"2024-05-11T01:16:32.111165+0000","last_fullsized":"2024-05-11T01:16:32.111165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:32.111043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:32.111043+0000","last_clean_scrub_stamp":"2024-05-11T01:16:32.111043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:03:33.670732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:28.355018+0000","last_change":"2024-05-11T01:16:28.355042+0000","last_active":"2024-05-11T01:16:28.355018+0000","last_peered":"2024-05-11T01:16:28.355018+0000","last_clean":"2024-05-11T01:16:28.355018+0000","last_became_active":"2024-05-11T01:13:59.445971+0000","last_became_peered":"2024-05-11T01:13:59.445971+0000","last_unstale":"2024-05-11T01:16:28.355018+0000","last_undegraded":"2024-05-11T01:16:28.355018+0000","last_fullsized":"2024-05-11T01:16:28.355018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:28.354929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:28.354929+0000","last_clean_scrub_stamp":"2024-05-11T01:16:28.354929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:34:59.581628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:25.374625+0000","last_change":"2024-05-11T01:16:25.374648+0000","last_active":"2024-05-11T01:16:25.374625+0000","last_peered":"2024-05-11T01:16:25.374625+0000","last_clean":"2024-05-11T01:16:25.374625+0000","last_became_active":"2024-05-11T01:13:59.430408+0000","last_became_peered":"2024-05-11T01:13:59.430408+0000","last_unstale":"2024-05-11T01:16:25.374625+0000","last_undegraded":"2024-05-11T01:16:25.374625+0000","last_fullsized":"2024-05-11T01:16:25.374625+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:25.374541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:25.374541+0000","last_clean_scrub_stamp":"2024-05-11T01:16:25.374541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:11:06.468338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:30.801913+0000","last_change":"2024-05-11T01:16:30.801936+0000","last_active":"2024-05-11T01:16:30.801913+0000","last_peered":"2024-05-11T01:16:30.801913+0000","last_clean":"2024-05-11T01:16:30.801913+0000","last_became_active":"2024-05-11T01:13:59.446986+0000","last_became_peered":"2024-05-11T01:13:59.446986+0000","last_unstale":"2024-05-11T01:16:30.801913+0000","last_undegraded":"2024-05-11T01:16:30.801913+0000","last_fullsized":"2024-05-11T01:16:30.801913+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:30.801807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:30.801807+0000","last_clean_scrub_stamp":"2024-05-11T01:16:30.801807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:40:44.369651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:29.357985+0000","last_change":"2024-05-11T01:16:29.357985+0000","last_active":"2024-05-11T01:16:29.357985+0000","last_peered":"2024-05-11T01:16:29.357985+0000","last_clean":"2024-05-11T01:16:29.357985+0000","last_became_active":"2024-05-11T01:13:59.445922+0000","last_became_peered":"2024-05-11T01:13:59.445922+0000","last_unstale":"2024-05-11T01:16:29.357985+0000","last_undegraded":"2024-05-11T01:16:29.357985+0000","last_fullsized":"2024-05-11T01:16:29.357985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:29.357867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:29.357867+0000","last_clean_scrub_stamp":"2024-05-11T01:16:29.357867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:43:20.378209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:28.411215+0000","last_change":"2024-05-11T01:16:28.411215+0000","last_active":"2024-05-11T01:16:28.411215+0000","last_peered":"2024-05-11T01:16:28.411215+0000","last_clean":"2024-05-11T01:16:28.411215+0000","last_became_active":"2024-05-11T01:13:59.434634+0000","last_became_peered":"2024-05-11T01:13:59.434634+0000","last_unstale":"2024-05-11T01:16:28.411215+0000","last_undegraded":"2024-05-11T01:16:28.411215+0000","last_fullsized":"2024-05-11T01:16:28.411215+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:28.411056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:28.411056+0000","last_clean_scrub_stamp":"2024-05-11T01:16:28.411056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:41:04.839118+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:31.114667+0000","last_change":"2024-05-11T01:16:31.114667+0000","last_active":"2024-05-11T01:16:31.114667+0000","last_peered":"2024-05-11T01:16:31.114667+0000","last_clean":"2024-05-11T01:16:31.114667+0000","last_became_active":"2024-05-11T01:13:59.433976+0000","last_became_peered":"2024-05-11T01:13:59.433976+0000","last_unstale":"2024-05-11T01:16:31.114667+0000","last_undegraded":"2024-05-11T01:16:31.114667+0000","last_fullsized":"2024-05-11T01:16:31.114667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:31.114492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:31.114492+0000","last_clean_scrub_stamp":"2024-05-11T01:16:31.114492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:12:48.867013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:29.812894+0000","last_change":"2024-05-11T01:16:29.812894+0000","last_active":"2024-05-11T01:16:29.812894+0000","last_peered":"2024-05-11T01:16:29.812894+0000","last_clean":"2024-05-11T01:16:29.812894+0000","last_became_active":"2024-05-11T01:13:59.446507+0000","last_became_peered":"2024-05-11T01:13:59.446507+0000","last_unstale":"2024-05-11T01:16:29.812894+0000","last_undegraded":"2024-05-11T01:16:29.812894+0000","last_fullsized":"2024-05-11T01:16:29.812894+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:29.812768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:29.812768+0000","last_clean_scrub_stamp":"2024-05-11T01:16:29.812768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:00.292156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:33.840692+0000","last_change":"2024-05-11T01:16:33.840721+0000","last_active":"2024-05-11T01:16:33.840692+0000","last_peered":"2024-05-11T01:16:33.840692+0000","last_clean":"2024-05-11T01:16:33.840692+0000","last_became_active":"2024-05-11T01:13:59.449670+0000","last_became_peered":"2024-05-11T01:13:59.449670+0000","last_unstale":"2024-05-11T01:16:33.840692+0000","last_undegraded":"2024-05-11T01:16:33.840692+0000","last_fullsized":"2024-05-11T01:16:33.840692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:33.840596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:33.840596+0000","last_clean_scrub_stamp":"2024-05-11T01:16:33.840596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:05:36.779913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:34.798138+0000","last_change":"2024-05-11T01:16:34.798179+0000","last_active":"2024-05-11T01:16:34.798138+0000","last_peered":"2024-05-11T01:16:34.798138+0000","last_clean":"2024-05-11T01:16:34.798138+0000","last_became_active":"2024-05-11T01:13:59.449822+0000","last_became_peered":"2024-05-11T01:13:59.449822+0000","last_unstale":"2024-05-11T01:16:34.798138+0000","last_undegraded":"2024-05-11T01:16:34.798138+0000","last_fullsized":"2024-05-11T01:16:34.798138+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:34.798023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:34.798023+0000","last_clean_scrub_stamp":"2024-05-11T01:16:34.798023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:56:32.443020+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:33.307750+0000","last_change":"2024-05-11T01:16:33.307774+0000","last_active":"2024-05-11T01:16:33.307750+0000","last_peered":"2024-05-11T01:16:33.307750+0000","last_clean":"2024-05-11T01:16:33.307750+0000","last_became_active":"2024-05-11T01:14:46.839837+0000","last_became_peered":"2024-05-11T01:14:46.839837+0000","last_unstale":"2024-05-11T01:16:33.307750+0000","last_undegraded":"2024-05-11T01:16:33.307750+0000","last_fullsized":"2024-05-11T01:16:33.307750+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:33.307628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:33.307628+0000","last_clean_scrub_stamp":"2024-05-11T01:16:33.307628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:42:51.602197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:25.372832+0000","last_change":"2024-05-11T01:16:25.372832+0000","last_active":"2024-05-11T01:16:25.372832+0000","last_peered":"2024-05-11T01:16:25.372832+0000","last_clean":"2024-05-11T01:16:25.372832+0000","last_became_active":"2024-05-11T01:13:59.428202+0000","last_became_peered":"2024-05-11T01:13:59.428202+0000","last_unstale":"2024-05-11T01:16:25.372832+0000","last_undegraded":"2024-05-11T01:16:25.372832+0000","last_fullsized":"2024-05-11T01:16:25.372832+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:25.372716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:25.372716+0000","last_clean_scrub_stamp":"2024-05-11T01:16:25.372716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:29:36.904518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:24.345778+0000","last_change":"2024-05-11T01:16:24.345778+0000","last_active":"2024-05-11T01:16:24.345778+0000","last_peered":"2024-05-11T01:16:24.345778+0000","last_clean":"2024-05-11T01:16:24.345778+0000","last_became_active":"2024-05-11T01:13:59.433324+0000","last_became_peered":"2024-05-11T01:13:59.433324+0000","last_unstale":"2024-05-11T01:16:24.345778+0000","last_undegraded":"2024-05-11T01:16:24.345778+0000","last_fullsized":"2024-05-11T01:16:24.345778+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:24.345618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:24.345618+0000","last_clean_scrub_stamp":"2024-05-11T01:16:24.345618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:12:14.641990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:18.676722+0000","last_change":"2024-05-11T01:16:18.676722+0000","last_active":"2024-05-11T01:16:18.676722+0000","last_peered":"2024-05-11T01:16:18.676722+0000","last_clean":"2024-05-11T01:16:18.676722+0000","last_became_active":"2024-05-11T01:13:59.446349+0000","last_became_peered":"2024-05-11T01:13:59.446349+0000","last_unstale":"2024-05-11T01:16:18.676722+0000","last_undegraded":"2024-05-11T01:16:18.676722+0000","last_fullsized":"2024-05-11T01:16:18.676722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:18.676611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:18.676611+0000","last_clean_scrub_stamp":"2024-05-11T01:16:18.676611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:52:47.351958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:33.441343+0000","last_change":"2024-05-11T01:16:33.441369+0000","last_active":"2024-05-11T01:16:33.441343+0000","last_peered":"2024-05-11T01:16:33.441343+0000","last_clean":"2024-05-11T01:16:33.441343+0000","last_became_active":"2024-05-11T01:13:59.427975+0000","last_became_peered":"2024-05-11T01:13:59.427975+0000","last_unstale":"2024-05-11T01:16:33.441343+0000","last_undegraded":"2024-05-11T01:16:33.441343+0000","last_fullsized":"2024-05-11T01:16:33.441343+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:33.441239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:33.441239+0000","last_clean_scrub_stamp":"2024-05-11T01:16:33.441239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:38.185239+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:28.863316+0000","last_change":"2024-05-11T01:16:28.863357+0000","last_active":"2024-05-11T01:16:28.863316+0000","last_peered":"2024-05-11T01:16:28.863316+0000","last_clean":"2024-05-11T01:16:28.863316+0000","last_became_active":"2024-05-11T01:13:59.446396+0000","last_became_peered":"2024-05-11T01:13:59.446396+0000","last_unstale":"2024-05-11T01:16:28.863316+0000","last_undegraded":"2024-05-11T01:16:28.863316+0000","last_fullsized":"2024-05-11T01:16:28.863316+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:28.863200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:28.863200+0000","last_clean_scrub_stamp":"2024-05-11T01:16:28.863200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:39:21.887258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:29.299867+0000","last_change":"2024-05-11T01:16:29.299898+0000","last_active":"2024-05-11T01:16:29.299867+0000","last_peered":"2024-05-11T01:16:29.299867+0000","last_clean":"2024-05-11T01:16:29.299867+0000","last_became_active":"2024-05-11T01:13:59.436992+0000","last_became_peered":"2024-05-11T01:13:59.436992+0000","last_unstale":"2024-05-11T01:16:29.299867+0000","last_undegraded":"2024-05-11T01:16:29.299867+0000","last_fullsized":"2024-05-11T01:16:29.299867+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:29.299758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:29.299758+0000","last_clean_scrub_stamp":"2024-05-11T01:16:29.299758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:56:14.903880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:30.331666+0000","last_change":"2024-05-11T01:16:30.331689+0000","last_active":"2024-05-11T01:16:30.331666+0000","last_peered":"2024-05-11T01:16:30.331666+0000","last_clean":"2024-05-11T01:16:30.331666+0000","last_became_active":"2024-05-11T01:13:59.445318+0000","last_became_peered":"2024-05-11T01:13:59.445318+0000","last_unstale":"2024-05-11T01:16:30.331666+0000","last_undegraded":"2024-05-11T01:16:30.331666+0000","last_fullsized":"2024-05-11T01:16:30.331666+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:30.331571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:30.331571+0000","last_clean_scrub_stamp":"2024-05-11T01:16:30.331571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:00:25.756081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:31.884539+0000","last_change":"2024-05-11T01:16:31.884565+0000","last_active":"2024-05-11T01:16:31.884539+0000","last_peered":"2024-05-11T01:16:31.884539+0000","last_clean":"2024-05-11T01:16:31.884539+0000","last_became_active":"2024-05-11T01:14:46.855320+0000","last_became_peered":"2024-05-11T01:14:46.855320+0000","last_unstale":"2024-05-11T01:16:31.884539+0000","last_undegraded":"2024-05-11T01:16:31.884539+0000","last_fullsized":"2024-05-11T01:16:31.884539+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:31.884442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:31.884442+0000","last_clean_scrub_stamp":"2024-05-11T01:16:31.884442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:40:45.452286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:26.311062+0000","last_change":"2024-05-11T01:16:26.311085+0000","last_active":"2024-05-11T01:16:26.311062+0000","last_peered":"2024-05-11T01:16:26.311062+0000","last_clean":"2024-05-11T01:16:26.311062+0000","last_became_active":"2024-05-11T01:13:59.443762+0000","last_became_peered":"2024-05-11T01:13:59.443762+0000","last_unstale":"2024-05-11T01:16:26.311062+0000","last_undegraded":"2024-05-11T01:16:26.311062+0000","last_fullsized":"2024-05-11T01:16:26.311062+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:26.310967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:26.310967+0000","last_clean_scrub_stamp":"2024-05-11T01:16:26.310967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:09:26.658356+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:32.440325+0000","last_change":"2024-05-11T01:16:32.440349+0000","last_active":"2024-05-11T01:16:32.440325+0000","last_peered":"2024-05-11T01:16:32.440325+0000","last_clean":"2024-05-11T01:16:32.440325+0000","last_became_active":"2024-05-11T01:13:59.437444+0000","last_became_peered":"2024-05-11T01:13:59.437444+0000","last_unstale":"2024-05-11T01:16:32.440325+0000","last_undegraded":"2024-05-11T01:16:32.440325+0000","last_fullsized":"2024-05-11T01:16:32.440325+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:32.440230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:32.440230+0000","last_clean_scrub_stamp":"2024-05-11T01:16:32.440230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:18:55.817200+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:28.825194+0000","last_change":"2024-05-11T01:16:28.825222+0000","last_active":"2024-05-11T01:16:28.825194+0000","last_peered":"2024-05-11T01:16:28.825194+0000","last_clean":"2024-05-11T01:16:28.825194+0000","last_became_active":"2024-05-11T01:13:59.446779+0000","last_became_peered":"2024-05-11T01:13:59.446779+0000","last_unstale":"2024-05-11T01:16:28.825194+0000","last_undegraded":"2024-05-11T01:16:28.825194+0000","last_fullsized":"2024-05-11T01:16:28.825194+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:28.825097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:28.825097+0000","last_clean_scrub_stamp":"2024-05-11T01:16:28.825097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:54:32.432363+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:20.932155+0000","last_change":"2024-05-11T01:16:20.932177+0000","last_active":"2024-05-11T01:16:20.932155+0000","last_peered":"2024-05-11T01:16:20.932155+0000","last_clean":"2024-05-11T01:16:20.932155+0000","last_became_active":"2024-05-11T01:13:59.431309+0000","last_became_peered":"2024-05-11T01:13:59.431309+0000","last_unstale":"2024-05-11T01:16:20.932155+0000","last_undegraded":"2024-05-11T01:16:20.932155+0000","last_fullsized":"2024-05-11T01:16:20.932155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:20.932060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:20.932060+0000","last_clean_scrub_stamp":"2024-05-11T01:16:20.932060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:01:03.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:23.953452+0000","last_change":"2024-05-11T01:16:23.953476+0000","last_active":"2024-05-11T01:16:23.953452+0000","last_peered":"2024-05-11T01:16:23.953452+0000","last_clean":"2024-05-11T01:16:23.953452+0000","last_became_active":"2024-05-11T01:13:59.433606+0000","last_became_peered":"2024-05-11T01:13:59.433606+0000","last_unstale":"2024-05-11T01:16:23.953452+0000","last_undegraded":"2024-05-11T01:16:23.953452+0000","last_fullsized":"2024-05-11T01:16:23.953452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:23.953361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:23.953361+0000","last_clean_scrub_stamp":"2024-05-11T01:16:23.953361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:41:52.718711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:24.954455+0000","last_change":"2024-05-11T01:16:24.954484+0000","last_active":"2024-05-11T01:16:24.954455+0000","last_peered":"2024-05-11T01:16:24.954455+0000","last_clean":"2024-05-11T01:16:24.954455+0000","last_became_active":"2024-05-11T01:13:59.435708+0000","last_became_peered":"2024-05-11T01:13:59.435708+0000","last_unstale":"2024-05-11T01:16:24.954455+0000","last_undegraded":"2024-05-11T01:16:24.954455+0000","last_fullsized":"2024-05-11T01:16:24.954455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:24.954343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:24.954343+0000","last_clean_scrub_stamp":"2024-05-11T01:16:24.954343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:06:49.984534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:25.332045+0000","last_change":"2024-05-11T01:16:25.332070+0000","last_active":"2024-05-11T01:16:25.332045+0000","last_peered":"2024-05-11T01:16:25.332045+0000","last_clean":"2024-05-11T01:16:25.332045+0000","last_became_active":"2024-05-11T01:13:59.440852+0000","last_became_peered":"2024-05-11T01:13:59.440852+0000","last_unstale":"2024-05-11T01:16:25.332045+0000","last_undegraded":"2024-05-11T01:16:25.332045+0000","last_fullsized":"2024-05-11T01:16:25.332045+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:25.331952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:25.331952+0000","last_clean_scrub_stamp":"2024-05-11T01:16:25.331952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:05:49.684778+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:28.347833+0000","last_change":"2024-05-11T01:16:28.347869+0000","last_active":"2024-05-11T01:16:28.347833+0000","last_peered":"2024-05-11T01:16:28.347833+0000","last_clean":"2024-05-11T01:16:28.347833+0000","last_became_active":"2024-05-11T01:13:59.436260+0000","last_became_peered":"2024-05-11T01:13:59.436260+0000","last_unstale":"2024-05-11T01:16:28.347833+0000","last_undegraded":"2024-05-11T01:16:28.347833+0000","last_fullsized":"2024-05-11T01:16:28.347833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:28.347748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:28.347748+0000","last_clean_scrub_stamp":"2024-05-11T01:16:28.347748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:33:23.835909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:24.373510+0000","last_change":"2024-05-11T01:16:24.373533+0000","last_active":"2024-05-11T01:16:24.373510+0000","last_peered":"2024-05-11T01:16:24.373510+0000","last_clean":"2024-05-11T01:16:24.373510+0000","last_became_active":"2024-05-11T01:13:59.424328+0000","last_became_peered":"2024-05-11T01:13:59.424328+0000","last_unstale":"2024-05-11T01:16:24.373510+0000","last_undegraded":"2024-05-11T01:16:24.373510+0000","last_fullsized":"2024-05-11T01:16:24.373510+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:24.373423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:24.373423+0000","last_clean_scrub_stamp":"2024-05-11T01:16:24.373423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:21:36.498512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:25.924034+0000","last_change":"2024-05-11T01:16:25.924077+0000","last_active":"2024-05-11T01:16:25.924034+0000","last_peered":"2024-05-11T01:16:25.924034+0000","last_clean":"2024-05-11T01:16:25.924034+0000","last_became_active":"2024-05-11T01:13:59.436257+0000","last_became_peered":"2024-05-11T01:13:59.436257+0000","last_unstale":"2024-05-11T01:16:25.924034+0000","last_undegraded":"2024-05-11T01:16:25.924034+0000","last_fullsized":"2024-05-11T01:16:25.924034+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:25.923929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:25.923929+0000","last_clean_scrub_stamp":"2024-05-11T01:16:25.923929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:34:26.969080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:27.358440+0000","last_change":"2024-05-11T01:16:27.358472+0000","last_active":"2024-05-11T01:16:27.358440+0000","last_peered":"2024-05-11T01:16:27.358440+0000","last_clean":"2024-05-11T01:16:27.358440+0000","last_became_active":"2024-05-11T01:13:59.435148+0000","last_became_peered":"2024-05-11T01:13:59.435148+0000","last_unstale":"2024-05-11T01:16:27.358440+0000","last_undegraded":"2024-05-11T01:16:27.358440+0000","last_fullsized":"2024-05-11T01:16:27.358440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:27.358346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:27.358346+0000","last_clean_scrub_stamp":"2024-05-11T01:16:27.358346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:06:51.517759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:34.433423+0000","last_change":"2024-05-11T01:16:34.433452+0000","last_active":"2024-05-11T01:16:34.433423+0000","last_peered":"2024-05-11T01:16:34.433423+0000","last_clean":"2024-05-11T01:16:34.433423+0000","last_became_active":"2024-05-11T01:13:59.438262+0000","last_became_peered":"2024-05-11T01:13:59.438262+0000","last_unstale":"2024-05-11T01:16:34.433423+0000","last_undegraded":"2024-05-11T01:16:34.433423+0000","last_fullsized":"2024-05-11T01:16:34.433423+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:34.433318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:34.433318+0000","last_clean_scrub_stamp":"2024-05-11T01:16:34.433318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:33:39.061542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:23.304890+0000","last_change":"2024-05-11T01:16:23.304918+0000","last_active":"2024-05-11T01:16:23.304890+0000","last_peered":"2024-05-11T01:16:23.304890+0000","last_clean":"2024-05-11T01:16:23.304890+0000","last_became_active":"2024-05-11T01:13:59.431691+0000","last_became_peered":"2024-05-11T01:13:59.431691+0000","last_unstale":"2024-05-11T01:16:23.304890+0000","last_undegraded":"2024-05-11T01:16:23.304890+0000","last_fullsized":"2024-05-11T01:16:23.304890+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:23.304773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:23.304773+0000","last_clean_scrub_stamp":"2024-05-11T01:16:23.304773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:22:15.763463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:23.884801+0000","last_change":"2024-05-11T01:16:23.884825+0000","last_active":"2024-05-11T01:16:23.884801+0000","last_peered":"2024-05-11T01:16:23.884801+0000","last_clean":"2024-05-11T01:16:23.884801+0000","last_became_active":"2024-05-11T01:13:59.440107+0000","last_became_peered":"2024-05-11T01:13:59.440107+0000","last_unstale":"2024-05-11T01:16:23.884801+0000","last_undegraded":"2024-05-11T01:16:23.884801+0000","last_fullsized":"2024-05-11T01:16:23.884801+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:23.884704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:23.884704+0000","last_clean_scrub_stamp":"2024-05-11T01:16:23.884704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:51:59.159799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"63'6","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:25.398599+0000","last_change":"2024-05-11T01:16:25.398628+0000","last_active":"2024-05-11T01:16:25.398599+0000","last_peered":"2024-05-11T01:16:25.398599+0000","last_clean":"2024-05-11T01:16:25.398599+0000","last_became_active":"2024-05-11T01:13:58.439841+0000","last_became_peered":"2024-05-11T01:13:58.439841+0000","last_unstale":"2024-05-11T01:16:25.398599+0000","last_undegraded":"2024-05-11T01:16:25.398599+0000","last_fullsized":"2024-05-11T01:16:25.398599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-05-11T01:16:25.398514+0000","last_deep_scrub":"63'6","last_deep_scrub_stamp":"2024-05-11T01:16:25.398514+0000","last_clean_scrub_stamp":"2024-05-11T01:16:25.398514+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:18:52.049723+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:22.922947+0000","last_change":"2024-05-11T01:16:22.922968+0000","last_active":"2024-05-11T01:16:22.922947+0000","last_peered":"2024-05-11T01:16:22.922947+0000","last_clean":"2024-05-11T01:16:22.922947+0000","last_became_active":"2024-05-11T01:13:59.433441+0000","last_became_peered":"2024-05-11T01:13:59.433441+0000","last_unstale":"2024-05-11T01:16:22.922947+0000","last_undegraded":"2024-05-11T01:16:22.922947+0000","last_fullsized":"2024-05-11T01:16:22.922947+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:22.922858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:22.922858+0000","last_clean_scrub_stamp":"2024-05-11T01:16:22.922858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:07:17.628558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":29,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:23.349822+0000","last_change":"2024-05-11T01:16:23.349860+0000","last_active":"2024-05-11T01:16:23.349822+0000","last_peered":"2024-05-11T01:16:23.349822+0000","last_clean":"2024-05-11T01:16:23.349822+0000","last_became_active":"2024-05-11T01:13:58.433649+0000","last_became_peered":"2024-05-11T01:13:58.433649+0000","last_unstale":"2024-05-11T01:16:23.349822+0000","last_undegraded":"2024-05-11T01:16:23.349822+0000","last_fullsized":"2024-05-11T01:16:23.349822+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-05-11T01:16:23.349692+0000","last_deep_scrub":"60'1","last_deep_scrub_stamp":"2024-05-11T01:16:23.349692+0000","last_clean_scrub_stamp":"2024-05-11T01:16:23.349692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:10:22.241262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:27.336905+0000","last_change":"2024-05-11T01:16:27.336905+0000","last_active":"2024-05-11T01:16:27.336905+0000","last_peered":"2024-05-11T01:16:27.336905+0000","last_clean":"2024-05-11T01:16:27.336905+0000","last_became_active":"2024-05-11T01:13:59.439073+0000","last_became_peered":"2024-05-11T01:13:59.439073+0000","last_unstale":"2024-05-11T01:16:27.336905+0000","last_undegraded":"2024-05-11T01:16:27.336905+0000","last_fullsized":"2024-05-11T01:16:27.336905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:27.336751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:27.336751+0000","last_clean_scrub_stamp":"2024-05-11T01:16:27.336751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:11:21.179123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"63'3","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:20.938966+0000","last_change":"2024-05-11T01:16:20.938966+0000","last_active":"2024-05-11T01:16:20.938966+0000","last_peered":"2024-05-11T01:16:20.938966+0000","last_clean":"2024-05-11T01:16:20.938966+0000","last_became_active":"2024-05-11T01:13:58.434345+0000","last_became_peered":"2024-05-11T01:13:58.434345+0000","last_unstale":"2024-05-11T01:16:20.938966+0000","last_undegraded":"2024-05-11T01:16:20.938966+0000","last_fullsized":"2024-05-11T01:16:20.938966+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-05-11T01:16:20.938870+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-05-11T01:16:20.938870+0000","last_clean_scrub_stamp":"2024-05-11T01:16:20.938870+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:47:09.458439+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:22.925289+0000","last_change":"2024-05-11T01:16:22.925289+0000","last_active":"2024-05-11T01:16:22.925289+0000","last_peered":"2024-05-11T01:16:22.925289+0000","last_clean":"2024-05-11T01:16:22.925289+0000","last_became_active":"2024-05-11T01:13:59.439153+0000","last_became_peered":"2024-05-11T01:13:59.439153+0000","last_unstale":"2024-05-11T01:16:22.925289+0000","last_undegraded":"2024-05-11T01:16:22.925289+0000","last_fullsized":"2024-05-11T01:16:22.925289+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:22.925127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:22.925127+0000","last_clean_scrub_stamp":"2024-05-11T01:16:22.925127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:09:45.833800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"63'3","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:24.252241+0000","last_change":"2024-05-11T01:16:24.252241+0000","last_active":"2024-05-11T01:16:24.252241+0000","last_peered":"2024-05-11T01:16:24.252241+0000","last_clean":"2024-05-11T01:16:24.252241+0000","last_became_active":"2024-05-11T01:13:58.433392+0000","last_became_peered":"2024-05-11T01:13:58.433392+0000","last_unstale":"2024-05-11T01:16:24.252241+0000","last_undegraded":"2024-05-11T01:16:24.252241+0000","last_fullsized":"2024-05-11T01:16:24.252241+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-05-11T01:16:24.252121+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-05-11T01:16:24.252121+0000","last_clean_scrub_stamp":"2024-05-11T01:16:24.252121+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:55.431496+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:26.379934+0000","last_change":"2024-05-11T01:16:26.379934+0000","last_active":"2024-05-11T01:16:26.379934+0000","last_peered":"2024-05-11T01:16:26.379934+0000","last_clean":"2024-05-11T01:16:26.379934+0000","last_became_active":"2024-05-11T01:13:59.434862+0000","last_became_peered":"2024-05-11T01:13:59.434862+0000","last_unstale":"2024-05-11T01:16:26.379934+0000","last_undegraded":"2024-05-11T01:16:26.379934+0000","last_fullsized":"2024-05-11T01:16:26.379934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:26.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:26.379801+0000","last_clean_scrub_stamp":"2024-05-11T01:16:26.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:38:21.009087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":29,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:23.413524+0000","last_change":"2024-05-11T01:16:23.413524+0000","last_active":"2024-05-11T01:16:23.413524+0000","last_peered":"2024-05-11T01:16:23.413524+0000","last_clean":"2024-05-11T01:16:23.413524+0000","last_became_active":"2024-05-11T01:13:58.439133+0000","last_became_peered":"2024-05-11T01:13:58.439133+0000","last_unstale":"2024-05-11T01:16:23.413524+0000","last_undegraded":"2024-05-11T01:16:23.413524+0000","last_fullsized":"2024-05-11T01:16:23.413524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-05-11T01:16:23.413341+0000","last_deep_scrub":"60'1","last_deep_scrub_stamp":"2024-05-11T01:16:23.413341+0000","last_clean_scrub_stamp":"2024-05-11T01:16:23.413341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:36:54.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:31.824833+0000","last_change":"2024-05-11T01:16:31.824869+0000","last_active":"2024-05-11T01:16:31.824833+0000","last_peered":"2024-05-11T01:16:31.824833+0000","last_clean":"2024-05-11T01:16:31.824833+0000","last_became_active":"2024-05-11T01:13:59.448208+0000","last_became_peered":"2024-05-11T01:13:59.448208+0000","last_unstale":"2024-05-11T01:16:31.824833+0000","last_undegraded":"2024-05-11T01:16:31.824833+0000","last_fullsized":"2024-05-11T01:16:31.824833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:31.824739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:31.824739+0000","last_clean_scrub_stamp":"2024-05-11T01:16:31.824739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:59:41.386146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":29,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:21.417783+0000","last_change":"2024-05-11T01:16:21.417783+0000","last_active":"2024-05-11T01:16:21.417783+0000","last_peered":"2024-05-11T01:16:21.417783+0000","last_clean":"2024-05-11T01:16:21.417783+0000","last_became_active":"2024-05-11T01:13:58.428556+0000","last_became_peered":"2024-05-11T01:13:58.428556+0000","last_unstale":"2024-05-11T01:16:21.417783+0000","last_undegraded":"2024-05-11T01:16:21.417783+0000","last_fullsized":"2024-05-11T01:16:21.417783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'1","last_scrub_stamp":"2024-05-11T01:16:21.417630+0000","last_deep_scrub":"60'1","last_deep_scrub_stamp":"2024-05-11T01:16:21.417630+0000","last_clean_scrub_stamp":"2024-05-11T01:16:21.417630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:51:56.692725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:19.702062+0000","last_change":"2024-05-11T01:16:19.702062+0000","last_active":"2024-05-11T01:16:19.702062+0000","last_peered":"2024-05-11T01:16:19.702062+0000","last_clean":"2024-05-11T01:16:19.702062+0000","last_became_active":"2024-05-11T01:13:59.446713+0000","last_became_peered":"2024-05-11T01:13:59.446713+0000","last_unstale":"2024-05-11T01:16:19.702062+0000","last_undegraded":"2024-05-11T01:16:19.702062+0000","last_fullsized":"2024-05-11T01:16:19.702062+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:19.701966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:19.701966+0000","last_clean_scrub_stamp":"2024-05-11T01:16:19.701966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:40:26.215605+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:21.303831+0000","last_change":"2024-05-11T01:16:21.303831+0000","last_active":"2024-05-11T01:16:21.303831+0000","last_peered":"2024-05-11T01:16:21.303831+0000","last_clean":"2024-05-11T01:16:21.303831+0000","last_became_active":"2024-05-11T01:13:58.427742+0000","last_became_peered":"2024-05-11T01:13:58.427742+0000","last_unstale":"2024-05-11T01:16:21.303831+0000","last_undegraded":"2024-05-11T01:16:21.303831+0000","last_fullsized":"2024-05-11T01:16:21.303831+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:21.303725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:21.303725+0000","last_clean_scrub_stamp":"2024-05-11T01:16:21.303725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:29:20.251109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:21.919960+0000","last_change":"2024-05-11T01:16:21.919960+0000","last_active":"2024-05-11T01:16:21.919960+0000","last_peered":"2024-05-11T01:16:21.919960+0000","last_clean":"2024-05-11T01:16:21.919960+0000","last_became_active":"2024-05-11T01:13:59.431292+0000","last_became_peered":"2024-05-11T01:13:59.431292+0000","last_unstale":"2024-05-11T01:16:21.919960+0000","last_undegraded":"2024-05-11T01:16:21.919960+0000","last_fullsized":"2024-05-11T01:16:21.919960+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:21.919793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:21.919793+0000","last_clean_scrub_stamp":"2024-05-11T01:16:21.919793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:17:06.944167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:24.709186+0000","last_change":"2024-05-11T01:16:24.709186+0000","last_active":"2024-05-11T01:16:24.709186+0000","last_peered":"2024-05-11T01:16:24.709186+0000","last_clean":"2024-05-11T01:16:24.709186+0000","last_became_active":"2024-05-11T01:13:58.435967+0000","last_became_peered":"2024-05-11T01:13:58.435967+0000","last_unstale":"2024-05-11T01:16:24.709186+0000","last_undegraded":"2024-05-11T01:16:24.709186+0000","last_fullsized":"2024-05-11T01:16:24.709186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:24.709073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:24.709073+0000","last_clean_scrub_stamp":"2024-05-11T01:16:24.709073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:34:25.754224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:31.361931+0000","last_change":"2024-05-11T01:16:31.361954+0000","last_active":"2024-05-11T01:16:31.361931+0000","last_peered":"2024-05-11T01:16:31.361931+0000","last_clean":"2024-05-11T01:16:31.361931+0000","last_became_active":"2024-05-11T01:13:59.445232+0000","last_became_peered":"2024-05-11T01:13:59.445232+0000","last_unstale":"2024-05-11T01:16:31.361931+0000","last_undegraded":"2024-05-11T01:16:31.361931+0000","last_fullsized":"2024-05-11T01:16:31.361931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:31.361805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:31.361805+0000","last_clean_scrub_stamp":"2024-05-11T01:16:31.361805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:15:51.550141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:20.385770+0000","last_change":"2024-05-11T01:16:20.385770+0000","last_active":"2024-05-11T01:16:20.385770+0000","last_peered":"2024-05-11T01:16:20.385770+0000","last_clean":"2024-05-11T01:16:20.385770+0000","last_became_active":"2024-05-11T01:13:58.425913+0000","last_became_peered":"2024-05-11T01:13:58.425913+0000","last_unstale":"2024-05-11T01:16:20.385770+0000","last_undegraded":"2024-05-11T01:16:20.385770+0000","last_fullsized":"2024-05-11T01:16:20.385770+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:20.385615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:20.385615+0000","last_clean_scrub_stamp":"2024-05-11T01:16:20.385615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:24:03.561234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:28.144003+0000","last_change":"2024-05-11T01:16:28.144028+0000","last_active":"2024-05-11T01:16:28.144003+0000","last_peered":"2024-05-11T01:16:28.144003+0000","last_clean":"2024-05-11T01:16:28.144003+0000","last_became_active":"2024-05-11T01:13:59.431126+0000","last_became_peered":"2024-05-11T01:13:59.431126+0000","last_unstale":"2024-05-11T01:16:28.144003+0000","last_undegraded":"2024-05-11T01:16:28.144003+0000","last_fullsized":"2024-05-11T01:16:28.144003+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:28.143894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:28.143894+0000","last_clean_scrub_stamp":"2024-05-11T01:16:28.143894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:37:33.480710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:22.674518+0000","last_change":"2024-05-11T01:16:22.674518+0000","last_active":"2024-05-11T01:16:22.674518+0000","last_peered":"2024-05-11T01:16:22.674518+0000","last_clean":"2024-05-11T01:16:22.674518+0000","last_became_active":"2024-05-11T01:13:58.431132+0000","last_became_peered":"2024-05-11T01:13:58.431132+0000","last_unstale":"2024-05-11T01:16:22.674518+0000","last_undegraded":"2024-05-11T01:16:22.674518+0000","last_fullsized":"2024-05-11T01:16:22.674518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:22.674347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:22.674347+0000","last_clean_scrub_stamp":"2024-05-11T01:16:22.674347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:20:51.635076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:30.413298+0000","last_change":"2024-05-11T01:16:30.413327+0000","last_active":"2024-05-11T01:16:30.413298+0000","last_peered":"2024-05-11T01:16:30.413298+0000","last_clean":"2024-05-11T01:16:30.413298+0000","last_became_active":"2024-05-11T01:13:59.436043+0000","last_became_peered":"2024-05-11T01:13:59.436043+0000","last_unstale":"2024-05-11T01:16:30.413298+0000","last_undegraded":"2024-05-11T01:16:30.413298+0000","last_fullsized":"2024-05-11T01:16:30.413298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:30.413207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:30.413207+0000","last_clean_scrub_stamp":"2024-05-11T01:16:30.413207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:12:48.165729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"63'5","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:25.209003+0000","last_change":"2024-05-11T01:16:25.209003+0000","last_active":"2024-05-11T01:16:25.209003+0000","last_peered":"2024-05-11T01:16:25.209003+0000","last_clean":"2024-05-11T01:16:25.209003+0000","last_became_active":"2024-05-11T01:13:58.435280+0000","last_became_peered":"2024-05-11T01:13:58.435280+0000","last_unstale":"2024-05-11T01:16:25.209003+0000","last_undegraded":"2024-05-11T01:16:25.209003+0000","last_fullsized":"2024-05-11T01:16:25.209003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-05-11T01:16:25.208819+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-05-11T01:16:25.208819+0000","last_clean_scrub_stamp":"2024-05-11T01:16:25.208819+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:10:28.328593+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:29.106984+0000","last_change":"2024-05-11T01:16:29.107010+0000","last_active":"2024-05-11T01:16:29.106984+0000","last_peered":"2024-05-11T01:16:29.106984+0000","last_clean":"2024-05-11T01:16:29.106984+0000","last_became_active":"2024-05-11T01:13:59.431913+0000","last_became_peered":"2024-05-11T01:13:59.431913+0000","last_unstale":"2024-05-11T01:16:29.106984+0000","last_undegraded":"2024-05-11T01:16:29.106984+0000","last_fullsized":"2024-05-11T01:16:29.106984+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:29.106871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:29.106871+0000","last_clean_scrub_stamp":"2024-05-11T01:16:29.106871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T01:41:05.534933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"63'12","reported_seq":40,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:19.308159+0000","last_change":"2024-05-11T01:16:19.308159+0000","last_active":"2024-05-11T01:16:19.308159+0000","last_peered":"2024-05-11T01:16:19.308159+0000","last_clean":"2024-05-11T01:16:19.308159+0000","last_became_active":"2024-05-11T01:13:58.424894+0000","last_became_peered":"2024-05-11T01:13:58.424894+0000","last_unstale":"2024-05-11T01:16:19.308159+0000","last_undegraded":"2024-05-11T01:16:19.308159+0000","last_fullsized":"2024-05-11T01:16:19.308159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-05-11T01:16:19.308054+0000","last_deep_scrub":"63'12","last_deep_scrub_stamp":"2024-05-11T01:16:19.308054+0000","last_clean_scrub_stamp":"2024-05-11T01:16:19.308054+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:39:19.175116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:29.837270+0000","last_change":"2024-05-11T01:16:29.837301+0000","last_active":"2024-05-11T01:16:29.837270+0000","last_peered":"2024-05-11T01:16:29.837270+0000","last_clean":"2024-05-11T01:16:29.837270+0000","last_became_active":"2024-05-11T01:13:59.447082+0000","last_became_peered":"2024-05-11T01:13:59.447082+0000","last_unstale":"2024-05-11T01:16:29.837270+0000","last_undegraded":"2024-05-11T01:16:29.837270+0000","last_fullsized":"2024-05-11T01:16:29.837270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:29.837176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:29.837176+0000","last_clean_scrub_stamp":"2024-05-11T01:16:29.837176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:54:33.444441+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"63'4","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:22.439653+0000","last_change":"2024-05-11T01:16:22.439702+0000","last_active":"2024-05-11T01:16:22.439653+0000","last_peered":"2024-05-11T01:16:22.439653+0000","last_clean":"2024-05-11T01:16:22.439653+0000","last_became_active":"2024-05-11T01:13:58.438603+0000","last_became_peered":"2024-05-11T01:13:58.438603+0000","last_unstale":"2024-05-11T01:16:22.439653+0000","last_undegraded":"2024-05-11T01:16:22.439653+0000","last_fullsized":"2024-05-11T01:16:22.439653+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-05-11T01:16:22.439559+0000","last_deep_scrub":"63'4","last_deep_scrub_stamp":"2024-05-11T01:16:22.439559+0000","last_clean_scrub_stamp":"2024-05-11T01:16:22.439559+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:19:58.130773+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:32.874348+0000","last_change":"2024-05-11T01:16:32.874372+0000","last_active":"2024-05-11T01:16:32.874348+0000","last_peered":"2024-05-11T01:16:32.874348+0000","last_clean":"2024-05-11T01:16:32.874348+0000","last_became_active":"2024-05-11T01:13:59.448532+0000","last_became_peered":"2024-05-11T01:13:59.448532+0000","last_unstale":"2024-05-11T01:16:32.874348+0000","last_undegraded":"2024-05-11T01:16:32.874348+0000","last_fullsized":"2024-05-11T01:16:32.874348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:32.874257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:32.874257+0000","last_clean_scrub_stamp":"2024-05-11T01:16:32.874257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:24.973752+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:23.236102+0000","last_change":"2024-05-11T01:16:23.236145+0000","last_active":"2024-05-11T01:16:23.236102+0000","last_peered":"2024-05-11T01:16:23.236102+0000","last_clean":"2024-05-11T01:16:23.236102+0000","last_became_active":"2024-05-11T01:13:58.433818+0000","last_became_peered":"2024-05-11T01:13:58.433818+0000","last_unstale":"2024-05-11T01:16:23.236102+0000","last_undegraded":"2024-05-11T01:16:23.236102+0000","last_fullsized":"2024-05-11T01:16:23.236102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:23.236003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:23.236003+0000","last_clean_scrub_stamp":"2024-05-11T01:16:23.236003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:11:17.078375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:27.386100+0000","last_change":"2024-05-11T01:16:27.386129+0000","last_active":"2024-05-11T01:16:27.386100+0000","last_peered":"2024-05-11T01:16:27.386100+0000","last_clean":"2024-05-11T01:16:27.386100+0000","last_became_active":"2024-05-11T01:13:59.434726+0000","last_became_peered":"2024-05-11T01:13:59.434726+0000","last_unstale":"2024-05-11T01:16:27.386100+0000","last_undegraded":"2024-05-11T01:16:27.386100+0000","last_fullsized":"2024-05-11T01:16:27.386100+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:27.386008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:27.386008+0000","last_clean_scrub_stamp":"2024-05-11T01:16:27.386008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:37:32.722823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:21.432782+0000","last_change":"2024-05-11T01:16:21.432782+0000","last_active":"2024-05-11T01:16:21.432782+0000","last_peered":"2024-05-11T01:16:21.432782+0000","last_clean":"2024-05-11T01:16:21.432782+0000","last_became_active":"2024-05-11T01:13:58.438528+0000","last_became_peered":"2024-05-11T01:13:58.438528+0000","last_unstale":"2024-05-11T01:16:21.432782+0000","last_undegraded":"2024-05-11T01:16:21.432782+0000","last_fullsized":"2024-05-11T01:16:21.432782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:21.432684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:21.432684+0000","last_clean_scrub_stamp":"2024-05-11T01:16:21.432684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:43:28.187080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:33.140743+0000","last_change":"2024-05-11T01:16:33.140766+0000","last_active":"2024-05-11T01:16:33.140743+0000","last_peered":"2024-05-11T01:16:33.140743+0000","last_clean":"2024-05-11T01:16:33.140743+0000","last_became_active":"2024-05-11T01:13:59.437176+0000","last_became_peered":"2024-05-11T01:13:59.437176+0000","last_unstale":"2024-05-11T01:16:33.140743+0000","last_undegraded":"2024-05-11T01:16:33.140743+0000","last_fullsized":"2024-05-11T01:16:33.140743+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:33.140650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:33.140650+0000","last_clean_scrub_stamp":"2024-05-11T01:16:33.140650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:18:56.517620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"63'12","reported_seq":40,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:21.690424+0000","last_change":"2024-05-11T01:16:21.690424+0000","last_active":"2024-05-11T01:16:21.690424+0000","last_peered":"2024-05-11T01:16:21.690424+0000","last_clean":"2024-05-11T01:16:21.690424+0000","last_became_active":"2024-05-11T01:13:58.430952+0000","last_became_peered":"2024-05-11T01:13:58.430952+0000","last_unstale":"2024-05-11T01:16:21.690424+0000","last_undegraded":"2024-05-11T01:16:21.690424+0000","last_fullsized":"2024-05-11T01:16:21.690424+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-05-11T01:16:21.690270+0000","last_deep_scrub":"63'12","last_deep_scrub_stamp":"2024-05-11T01:16:21.690270+0000","last_clean_scrub_stamp":"2024-05-11T01:16:21.690270+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T05:50:40.675254+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:21.309902+0000","last_change":"2024-05-11T01:16:21.309941+0000","last_active":"2024-05-11T01:16:21.309902+0000","last_peered":"2024-05-11T01:16:21.309902+0000","last_clean":"2024-05-11T01:16:21.309902+0000","last_became_active":"2024-05-11T01:13:59.426727+0000","last_became_peered":"2024-05-11T01:13:59.426727+0000","last_unstale":"2024-05-11T01:16:21.309902+0000","last_undegraded":"2024-05-11T01:16:21.309902+0000","last_fullsized":"2024-05-11T01:16:21.309902+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:21.309783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:21.309783+0000","last_clean_scrub_stamp":"2024-05-11T01:16:21.309783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:12:12.750322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"63'2","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:20.318302+0000","last_change":"2024-05-11T01:16:20.318329+0000","last_active":"2024-05-11T01:16:20.318302+0000","last_peered":"2024-05-11T01:16:20.318302+0000","last_clean":"2024-05-11T01:16:20.318302+0000","last_became_active":"2024-05-11T01:13:58.434215+0000","last_became_peered":"2024-05-11T01:13:58.434215+0000","last_unstale":"2024-05-11T01:16:20.318302+0000","last_undegraded":"2024-05-11T01:16:20.318302+0000","last_fullsized":"2024-05-11T01:16:20.318302+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-05-11T01:16:20.318225+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-05-11T01:16:20.318225+0000","last_clean_scrub_stamp":"2024-05-11T01:16:20.318225+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:21:46.728553+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:26.803964+0000","last_change":"2024-05-11T01:16:26.803989+0000","last_active":"2024-05-11T01:16:26.803964+0000","last_peered":"2024-05-11T01:16:26.803964+0000","last_clean":"2024-05-11T01:16:26.803964+0000","last_became_active":"2024-05-11T01:13:59.443981+0000","last_became_peered":"2024-05-11T01:13:59.443981+0000","last_unstale":"2024-05-11T01:16:26.803964+0000","last_undegraded":"2024-05-11T01:16:26.803964+0000","last_fullsized":"2024-05-11T01:16:26.803964+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:26.803851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:26.803851+0000","last_clean_scrub_stamp":"2024-05-11T01:16:26.803851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:41.265126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"63'3","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:22.370610+0000","last_change":"2024-05-11T01:16:22.370668+0000","last_active":"2024-05-11T01:16:22.370610+0000","last_peered":"2024-05-11T01:16:22.370610+0000","last_clean":"2024-05-11T01:16:22.370610+0000","last_became_active":"2024-05-11T01:13:58.429538+0000","last_became_peered":"2024-05-11T01:13:58.429538+0000","last_unstale":"2024-05-11T01:16:22.370610+0000","last_undegraded":"2024-05-11T01:16:22.370610+0000","last_fullsized":"2024-05-11T01:16:22.370610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-05-11T01:16:22.370511+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-05-11T01:16:22.370511+0000","last_clean_scrub_stamp":"2024-05-11T01:16:22.370511+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:34:14.166587+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:27.887655+0000","last_change":"2024-05-11T01:16:27.887695+0000","last_active":"2024-05-11T01:16:27.887655+0000","last_peered":"2024-05-11T01:16:27.887655+0000","last_clean":"2024-05-11T01:16:27.887655+0000","last_became_active":"2024-05-11T01:13:59.438474+0000","last_became_peered":"2024-05-11T01:13:59.438474+0000","last_unstale":"2024-05-11T01:16:27.887655+0000","last_undegraded":"2024-05-11T01:16:27.887655+0000","last_fullsized":"2024-05-11T01:16:27.887655+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:27.887551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:27.887551+0000","last_clean_scrub_stamp":"2024-05-11T01:16:27.887551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:42.348827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"63'2","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:20.265140+0000","last_change":"2024-05-11T01:16:20.265169+0000","last_active":"2024-05-11T01:16:20.265140+0000","last_peered":"2024-05-11T01:16:20.265140+0000","last_clean":"2024-05-11T01:16:20.265140+0000","last_became_active":"2024-05-11T01:13:58.426586+0000","last_became_peered":"2024-05-11T01:13:58.426586+0000","last_unstale":"2024-05-11T01:16:20.265140+0000","last_undegraded":"2024-05-11T01:16:20.265140+0000","last_fullsized":"2024-05-11T01:16:20.265140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-05-11T01:16:20.265058+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-05-11T01:16:20.265058+0000","last_clean_scrub_stamp":"2024-05-11T01:16:20.265058+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T02:55:26.631399+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:26.925509+0000","last_change":"2024-05-11T01:16:26.925532+0000","last_active":"2024-05-11T01:16:26.925509+0000","last_peered":"2024-05-11T01:16:26.925509+0000","last_clean":"2024-05-11T01:16:26.925509+0000","last_became_active":"2024-05-11T01:13:59.436373+0000","last_became_peered":"2024-05-11T01:13:59.436373+0000","last_unstale":"2024-05-11T01:16:26.925509+0000","last_undegraded":"2024-05-11T01:16:26.925509+0000","last_fullsized":"2024-05-11T01:16:26.925509+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:26.925415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:26.925415+0000","last_clean_scrub_stamp":"2024-05-11T01:16:26.925415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:55:05.848616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:24.395786+0000","last_change":"2024-05-11T01:16:24.395811+0000","last_active":"2024-05-11T01:16:24.395786+0000","last_peered":"2024-05-11T01:16:24.395786+0000","last_clean":"2024-05-11T01:16:24.395786+0000","last_became_active":"2024-05-11T01:13:58.439404+0000","last_became_peered":"2024-05-11T01:13:58.439404+0000","last_unstale":"2024-05-11T01:16:24.395786+0000","last_undegraded":"2024-05-11T01:16:24.395786+0000","last_fullsized":"2024-05-11T01:16:24.395786+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:24.395700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:24.395700+0000","last_clean_scrub_stamp":"2024-05-11T01:16:24.395700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:10:27.515474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:29.375299+0000","last_change":"2024-05-11T01:16:29.375344+0000","last_active":"2024-05-11T01:16:29.375299+0000","last_peered":"2024-05-11T01:16:29.375299+0000","last_clean":"2024-05-11T01:16:29.375299+0000","last_became_active":"2024-05-11T01:13:59.435253+0000","last_became_peered":"2024-05-11T01:13:59.435253+0000","last_unstale":"2024-05-11T01:16:29.375299+0000","last_undegraded":"2024-05-11T01:16:29.375299+0000","last_fullsized":"2024-05-11T01:16:29.375299+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:29.375196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:29.375196+0000","last_clean_scrub_stamp":"2024-05-11T01:16:29.375196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:55:05.416323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"63'3","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:20.453431+0000","last_change":"2024-05-11T01:16:20.453454+0000","last_active":"2024-05-11T01:16:20.453431+0000","last_peered":"2024-05-11T01:16:20.453431+0000","last_clean":"2024-05-11T01:16:20.453431+0000","last_became_active":"2024-05-11T01:13:58.437597+0000","last_became_peered":"2024-05-11T01:13:58.437597+0000","last_unstale":"2024-05-11T01:16:20.453431+0000","last_undegraded":"2024-05-11T01:16:20.453431+0000","last_fullsized":"2024-05-11T01:16:20.453431+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-05-11T01:16:20.453363+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-05-11T01:16:20.453363+0000","last_clean_scrub_stamp":"2024-05-11T01:16:20.453363+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:21:32.578452+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:21.905755+0000","last_change":"2024-05-11T01:16:21.905778+0000","last_active":"2024-05-11T01:16:21.905755+0000","last_peered":"2024-05-11T01:16:21.905755+0000","last_clean":"2024-05-11T01:16:21.905755+0000","last_became_active":"2024-05-11T01:13:59.431386+0000","last_became_peered":"2024-05-11T01:13:59.431386+0000","last_unstale":"2024-05-11T01:16:21.905755+0000","last_undegraded":"2024-05-11T01:16:21.905755+0000","last_fullsized":"2024-05-11T01:16:21.905755+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:21.905667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:21.905667+0000","last_clean_scrub_stamp":"2024-05-11T01:16:21.905667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:27:56.256157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:22.300343+0000","last_change":"2024-05-11T01:16:22.300364+0000","last_active":"2024-05-11T01:16:22.300343+0000","last_peered":"2024-05-11T01:16:22.300343+0000","last_clean":"2024-05-11T01:16:22.300343+0000","last_became_active":"2024-05-11T01:13:58.430890+0000","last_became_peered":"2024-05-11T01:13:58.430890+0000","last_unstale":"2024-05-11T01:16:22.300343+0000","last_undegraded":"2024-05-11T01:16:22.300343+0000","last_fullsized":"2024-05-11T01:16:22.300343+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:22.300235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:22.300235+0000","last_clean_scrub_stamp":"2024-05-11T01:16:22.300235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:13:56.619799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:22.316411+0000","last_change":"2024-05-11T01:16:22.316446+0000","last_active":"2024-05-11T01:16:22.316411+0000","last_peered":"2024-05-11T01:16:22.316411+0000","last_clean":"2024-05-11T01:16:22.316411+0000","last_became_active":"2024-05-11T01:13:59.429734+0000","last_became_peered":"2024-05-11T01:13:59.429734+0000","last_unstale":"2024-05-11T01:16:22.316411+0000","last_undegraded":"2024-05-11T01:16:22.316411+0000","last_fullsized":"2024-05-11T01:16:22.316411+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:22.316298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:22.316298+0000","last_clean_scrub_stamp":"2024-05-11T01:16:22.316298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:36:32.221385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:20.671290+0000","last_change":"2024-05-11T01:16:20.671314+0000","last_active":"2024-05-11T01:16:20.671290+0000","last_peered":"2024-05-11T01:16:20.671290+0000","last_clean":"2024-05-11T01:16:20.671290+0000","last_became_active":"2024-05-11T01:13:58.430846+0000","last_became_peered":"2024-05-11T01:13:58.430846+0000","last_unstale":"2024-05-11T01:16:20.671290+0000","last_undegraded":"2024-05-11T01:16:20.671290+0000","last_fullsized":"2024-05-11T01:16:20.671290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:20.671213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:20.671213+0000","last_clean_scrub_stamp":"2024-05-11T01:16:20.671213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:16:25.639951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:35.751077+0000","last_change":"2024-05-11T01:16:35.751104+0000","last_active":"2024-05-11T01:16:35.751077+0000","last_peered":"2024-05-11T01:16:35.751077+0000","last_clean":"2024-05-11T01:16:35.751077+0000","last_became_active":"2024-05-11T01:13:59.450300+0000","last_became_peered":"2024-05-11T01:13:59.450300+0000","last_unstale":"2024-05-11T01:16:35.751077+0000","last_undegraded":"2024-05-11T01:16:35.751077+0000","last_fullsized":"2024-05-11T01:16:35.751077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:35.750979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:35.750979+0000","last_clean_scrub_stamp":"2024-05-11T01:16:35.750979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:51:15.211781+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:26.163073+0000","last_change":"2024-05-11T01:16:26.163098+0000","last_active":"2024-05-11T01:16:26.163073+0000","last_peered":"2024-05-11T01:16:26.163073+0000","last_clean":"2024-05-11T01:16:26.163073+0000","last_became_active":"2024-05-11T01:13:58.434702+0000","last_became_peered":"2024-05-11T01:13:58.434702+0000","last_unstale":"2024-05-11T01:16:26.163073+0000","last_undegraded":"2024-05-11T01:16:26.163073+0000","last_fullsized":"2024-05-11T01:16:26.163073+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:26.162972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:26.162972+0000","last_clean_scrub_stamp":"2024-05-11T01:16:26.162972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:18:52.814182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:30.870211+0000","last_change":"2024-05-11T01:16:30.870234+0000","last_active":"2024-05-11T01:16:30.870211+0000","last_peered":"2024-05-11T01:16:30.870211+0000","last_clean":"2024-05-11T01:16:30.870211+0000","last_became_active":"2024-05-11T01:13:59.451773+0000","last_became_peered":"2024-05-11T01:13:59.451773+0000","last_unstale":"2024-05-11T01:16:30.870211+0000","last_undegraded":"2024-05-11T01:16:30.870211+0000","last_fullsized":"2024-05-11T01:16:30.870211+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:30.870128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:30.870128+0000","last_clean_scrub_stamp":"2024-05-11T01:16:30.870128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:01.349516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"63'2","reported_seq":30,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:19.425513+0000","last_change":"2024-05-11T01:16:19.425564+0000","last_active":"2024-05-11T01:16:19.425513+0000","last_peered":"2024-05-11T01:16:19.425513+0000","last_clean":"2024-05-11T01:16:19.425513+0000","last_became_active":"2024-05-11T01:13:58.436650+0000","last_became_peered":"2024-05-11T01:13:58.436650+0000","last_unstale":"2024-05-11T01:16:19.425513+0000","last_undegraded":"2024-05-11T01:16:19.425513+0000","last_fullsized":"2024-05-11T01:16:19.425513+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-05-11T01:16:19.425401+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-05-11T01:16:19.425401+0000","last_clean_scrub_stamp":"2024-05-11T01:16:19.425401+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:32.097337+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:25.755771+0000","last_change":"2024-05-11T01:16:25.755793+0000","last_active":"2024-05-11T01:16:25.755771+0000","last_peered":"2024-05-11T01:16:25.755771+0000","last_clean":"2024-05-11T01:16:25.755771+0000","last_became_active":"2024-05-11T01:13:59.442413+0000","last_became_peered":"2024-05-11T01:13:59.442413+0000","last_unstale":"2024-05-11T01:16:25.755771+0000","last_undegraded":"2024-05-11T01:16:25.755771+0000","last_fullsized":"2024-05-11T01:16:25.755771+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:25.755682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:25.755682+0000","last_clean_scrub_stamp":"2024-05-11T01:16:25.755682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:55:04.678883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:27.176238+0000","last_change":"2024-05-11T01:16:27.176260+0000","last_active":"2024-05-11T01:16:27.176238+0000","last_peered":"2024-05-11T01:16:27.176238+0000","last_clean":"2024-05-11T01:16:27.176238+0000","last_became_active":"2024-05-11T01:13:58.436037+0000","last_became_peered":"2024-05-11T01:13:58.436037+0000","last_unstale":"2024-05-11T01:16:27.176238+0000","last_undegraded":"2024-05-11T01:16:27.176238+0000","last_fullsized":"2024-05-11T01:16:27.176238+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:27.176141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:27.176141+0000","last_clean_scrub_stamp":"2024-05-11T01:16:27.176141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:38.384818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:30.087748+0000","last_change":"2024-05-11T01:16:30.087780+0000","last_active":"2024-05-11T01:16:30.087748+0000","last_peered":"2024-05-11T01:16:30.087748+0000","last_clean":"2024-05-11T01:16:30.087748+0000","last_became_active":"2024-05-11T01:13:59.433946+0000","last_became_peered":"2024-05-11T01:13:59.433946+0000","last_unstale":"2024-05-11T01:16:30.087748+0000","last_undegraded":"2024-05-11T01:16:30.087748+0000","last_fullsized":"2024-05-11T01:16:30.087748+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:30.087616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:30.087616+0000","last_clean_scrub_stamp":"2024-05-11T01:16:30.087616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:55:06.128743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:18.904488+0000","last_change":"2024-05-11T01:16:18.904540+0000","last_active":"2024-05-11T01:16:18.904488+0000","last_peered":"2024-05-11T01:16:18.904488+0000","last_clean":"2024-05-11T01:16:18.904488+0000","last_became_active":"2024-05-11T01:13:58.432149+0000","last_became_peered":"2024-05-11T01:13:58.432149+0000","last_unstale":"2024-05-11T01:16:18.904488+0000","last_undegraded":"2024-05-11T01:16:18.904488+0000","last_fullsized":"2024-05-11T01:16:18.904488+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:18.904398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:18.904398+0000","last_clean_scrub_stamp":"2024-05-11T01:16:18.904398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:29:43.994446+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:31.398053+0000","last_change":"2024-05-11T01:16:31.398078+0000","last_active":"2024-05-11T01:16:31.398053+0000","last_peered":"2024-05-11T01:16:31.398053+0000","last_clean":"2024-05-11T01:16:31.398053+0000","last_became_active":"2024-05-11T01:13:59.432058+0000","last_became_peered":"2024-05-11T01:13:59.432058+0000","last_unstale":"2024-05-11T01:16:31.398053+0000","last_undegraded":"2024-05-11T01:16:31.398053+0000","last_fullsized":"2024-05-11T01:16:31.398053+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:31.397963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:31.397963+0000","last_clean_scrub_stamp":"2024-05-11T01:16:31.397963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:03:32.957652+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:22.223748+0000","last_change":"2024-05-11T01:16:22.223783+0000","last_active":"2024-05-11T01:16:22.223748+0000","last_peered":"2024-05-11T01:16:22.223748+0000","last_clean":"2024-05-11T01:16:22.223748+0000","last_became_active":"2024-05-11T01:13:58.433584+0000","last_became_peered":"2024-05-11T01:13:58.433584+0000","last_unstale":"2024-05-11T01:16:22.223748+0000","last_undegraded":"2024-05-11T01:16:22.223748+0000","last_fullsized":"2024-05-11T01:16:22.223748+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:22.223652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:22.223652+0000","last_clean_scrub_stamp":"2024-05-11T01:16:22.223652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:43:28.978047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:26.397360+0000","last_change":"2024-05-11T01:16:26.397381+0000","last_active":"2024-05-11T01:16:26.397360+0000","last_peered":"2024-05-11T01:16:26.397360+0000","last_clean":"2024-05-11T01:16:26.397360+0000","last_became_active":"2024-05-11T01:13:59.432337+0000","last_became_peered":"2024-05-11T01:13:59.432337+0000","last_unstale":"2024-05-11T01:16:26.397360+0000","last_undegraded":"2024-05-11T01:16:26.397360+0000","last_fullsized":"2024-05-11T01:16:26.397360+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:26.397266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:26.397266+0000","last_clean_scrub_stamp":"2024-05-11T01:16:26.397266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:37.605942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:24.353759+0000","last_change":"2024-05-11T01:16:24.353783+0000","last_active":"2024-05-11T01:16:24.353759+0000","last_peered":"2024-05-11T01:16:24.353759+0000","last_clean":"2024-05-11T01:16:24.353759+0000","last_became_active":"2024-05-11T01:13:58.434956+0000","last_became_peered":"2024-05-11T01:13:58.434956+0000","last_unstale":"2024-05-11T01:16:24.353759+0000","last_undegraded":"2024-05-11T01:16:24.353759+0000","last_fullsized":"2024-05-11T01:16:24.353759+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:24.353659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:24.353659+0000","last_clean_scrub_stamp":"2024-05-11T01:16:24.353659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T04:43:36.619665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:26.378603+0000","last_change":"2024-05-11T01:16:26.378666+0000","last_active":"2024-05-11T01:16:26.378603+0000","last_peered":"2024-05-11T01:16:26.378603+0000","last_clean":"2024-05-11T01:16:26.378603+0000","last_became_active":"2024-05-11T01:13:59.432012+0000","last_became_peered":"2024-05-11T01:13:59.432012+0000","last_unstale":"2024-05-11T01:16:26.378603+0000","last_undegraded":"2024-05-11T01:16:26.378603+0000","last_fullsized":"2024-05-11T01:16:26.378603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:26.378498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:26.378498+0000","last_clean_scrub_stamp":"2024-05-11T01:16:26.378498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T07:04:23.380095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:19.907575+0000","last_change":"2024-05-11T01:16:19.907597+0000","last_active":"2024-05-11T01:16:19.907575+0000","last_peered":"2024-05-11T01:16:19.907575+0000","last_clean":"2024-05-11T01:16:19.907575+0000","last_became_active":"2024-05-11T01:13:58.433411+0000","last_became_peered":"2024-05-11T01:13:58.433411+0000","last_unstale":"2024-05-11T01:16:19.907575+0000","last_undegraded":"2024-05-11T01:16:19.907575+0000","last_fullsized":"2024-05-11T01:16:19.907575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:19.907498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:19.907498+0000","last_clean_scrub_stamp":"2024-05-11T01:16:19.907498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:07:48.604811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:32.352728+0000","last_change":"2024-05-11T01:16:32.352767+0000","last_active":"2024-05-11T01:16:32.352728+0000","last_peered":"2024-05-11T01:16:32.352728+0000","last_clean":"2024-05-11T01:16:32.352728+0000","last_became_active":"2024-05-11T01:13:59.446670+0000","last_became_peered":"2024-05-11T01:13:59.446670+0000","last_unstale":"2024-05-11T01:16:32.352728+0000","last_undegraded":"2024-05-11T01:16:32.352728+0000","last_fullsized":"2024-05-11T01:16:32.352728+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:32.352605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:32.352605+0000","last_clean_scrub_stamp":"2024-05-11T01:16:32.352605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T03:12:20.268231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:23.684709+0000","last_change":"2024-05-11T01:16:23.684732+0000","last_active":"2024-05-11T01:16:23.684709+0000","last_peered":"2024-05-11T01:16:23.684709+0000","last_clean":"2024-05-11T01:16:23.684709+0000","last_became_active":"2024-05-11T01:13:58.434062+0000","last_became_peered":"2024-05-11T01:13:58.434062+0000","last_unstale":"2024-05-11T01:16:23.684709+0000","last_undegraded":"2024-05-11T01:16:23.684709+0000","last_fullsized":"2024-05-11T01:16:23.684709+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:23.684611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:23.684611+0000","last_clean_scrub_stamp":"2024-05-11T01:16:23.684611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:06:48.714802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:27.815646+0000","last_change":"2024-05-11T01:16:27.815671+0000","last_active":"2024-05-11T01:16:27.815646+0000","last_peered":"2024-05-11T01:16:27.815646+0000","last_clean":"2024-05-11T01:16:27.815646+0000","last_became_active":"2024-05-11T01:13:59.445146+0000","last_became_peered":"2024-05-11T01:13:59.445146+0000","last_unstale":"2024-05-11T01:16:27.815646+0000","last_undegraded":"2024-05-11T01:16:27.815646+0000","last_fullsized":"2024-05-11T01:16:27.815646+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T01:16:27.815554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T01:16:27.815554+0000","last_clean_scrub_stamp":"2024-05-11T01:16:27.815554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T06:39:20.839612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"63'3","reported_seq":31,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:23.344043+0000","last_change":"2024-05-11T01:16:23.344072+0000","last_active":"2024-05-11T01:16:23.344043+0000","last_peered":"2024-05-11T01:16:23.344043+0000","last_clean":"2024-05-11T01:16:23.344043+0000","last_became_active":"2024-05-11T01:13:58.434320+0000","last_became_peered":"2024-05-11T01:13:58.434320+0000","last_unstale":"2024-05-11T01:16:23.344043+0000","last_undegraded":"2024-05-11T01:16:23.344043+0000","last_fullsized":"2024-05-11T01:16:23.344043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-05-11T01:16:23.343961+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-05-11T01:16:23.343961+0000","last_clean_scrub_stamp":"2024-05-11T01:16:23.343961+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:42:36.015896+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"63'88","reported_seq":257,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-05-11T01:16:21.275988+0000","last_change":"2024-05-11T01:16:21.276029+0000","last_active":"2024-05-11T01:16:21.275988+0000","last_peered":"2024-05-11T01:16:21.275988+0000","last_clean":"2024-05-11T01:16:21.275988+0000","last_became_active":"2024-05-11T01:00:57.062819+0000","last_became_peered":"2024-05-11T01:00:57.062819+0000","last_unstale":"2024-05-11T01:16:21.275988+0000","last_undegraded":"2024-05-11T01:16:21.275988+0000","last_fullsized":"2024-05-11T01:16:21.275988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'88","last_scrub_stamp":"2024-05-11T01:16:21.275905+0000","last_deep_scrub":"63'88","last_deep_scrub_stamp":"2024-05-11T01:16:21.275905+0000","last_clean_scrub_stamp":"2024-05-11T01:16:21.275905+0000","objects_scrubbed":2,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:13:21.160128+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":3807,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1347584,"data_stored":949409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542336,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":757760,"data_stored":359623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54128,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317712,"statfs":{"total":96636764160,"available":96581337088,"internally_reserved":0,"allocated":761856,"data_stored":361960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":359589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575039,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":765952,"data_stored":361982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58280,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313560,"statfs":{"total":96636764160,"available":96577085440,"internally_reserved":0,"allocated":753664,"data_stored":359021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36652,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335188,"statfs":{"total":96636764160,"available":96599232512,"internally_reserved":0,"allocated":1347584,"data_stored":949353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T01:16:41.708 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-11T01:16:41.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.847+0000 7fa958b58640 1 Processor -- start 2024-05-11T01:16:41.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.848+0000 7fa958b58640 1 -- start start 2024-05-11T01:16:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.848+0000 7fa958b58640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa95415f2f0 0x7fa9540cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.848+0000 7fa958b58640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d2d60 0x7fa9540d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.848+0000 7fa958b58640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9540d3a90 con 0x7fa9540d4e40 2024-05-11T01:16:41.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.848+0000 7fa958b58640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa9540d3d70 con 0x7fa95415f2f0 2024-05-11T01:16:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.848+0000 7fa958b58640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa9540d4050 con 0x7fa9540d2d60 2024-05-11T01:16:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa95259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa95415f2f0 0x7fa9540cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa952d9d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d2d60 0x7fa9540d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa951d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa9540d4e40 0x7fa9540d5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:58706/0 (socket says 172.21.15.52:58706) 2024-05-11T01:16:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa951d9b640 1 -- 172.21.15.52:0/2805134206 learned_addr learned my addr 172.21.15.52:0/2805134206 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa95259c640 1 -- 172.21.15.52:0/2805134206 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d2d60 msgr2=0x7fa9540d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa95259c640 1 --2- 172.21.15.52:0/2805134206 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d2d60 0x7fa9540d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa95259c640 1 -- 172.21.15.52:0/2805134206 >> v1:172.21.15.119:6789/0 conn(0x7fa9540d4e40 legacy=0x7fa9540d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa95159a640 1 -- 172.21.15.52:0/2805134206 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638281990 0 0) 0x7fa9540d3a90 con 0x7fa9540d4e40 2024-05-11T01:16:41.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa95259c640 1 -- 172.21.15.52:0/2805134206 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9540b4e60 con 0x7fa95415f2f0 2024-05-11T01:16:41.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.849+0000 7fa952d9d640 1 --2- 172.21.15.52:0/2805134206 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d2d60 0x7fa9540d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:16:41.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.850+0000 7fa95259c640 1 --2- 172.21.15.52:0/2805134206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95415f2f0 0x7fa9540cfca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa944009d90 tx=0x7fa94402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8baf7eb2ae9251ec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:41.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.850+0000 7fa95159a640 1 -- 172.21.15.52:0/2805134206 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944007740 con 0x7fa95415f2f0 2024-05-11T01:16:41.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.850+0000 7fa95159a640 1 -- 172.21.15.52:0/2805134206 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9440078e0 con 0x7fa95415f2f0 2024-05-11T01:16:41.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.850+0000 7fa95159a640 1 -- 172.21.15.52:0/2805134206 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944007cb0 con 0x7fa95415f2f0 2024-05-11T01:16:41.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.851+0000 7fa958b58640 1 -- 172.21.15.52:0/2805134206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95415f2f0 msgr2=0x7fa9540cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.851+0000 7fa958b58640 1 --2- 172.21.15.52:0/2805134206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95415f2f0 0x7fa9540cfca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa944009d90 tx=0x7fa94402a660 comp rx=0 tx=0).stop 2024-05-11T01:16:41.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.851+0000 7fa958b58640 1 -- 172.21.15.52:0/2805134206 shutdown_connections 2024-05-11T01:16:41.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.851+0000 7fa958b58640 1 --2- 172.21.15.52:0/2805134206 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d2d60 0x7fa9540d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.851+0000 7fa958b58640 1 --2- 172.21.15.52:0/2805134206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95415f2f0 0x7fa9540cfca0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.851+0000 7fa958b58640 1 -- 172.21.15.52:0/2805134206 >> 172.21.15.52:0/2805134206 conn(0x7fa95408ebb0 msgr2=0x7fa95409e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:41.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.851+0000 7fa958b58640 1 -- 172.21.15.52:0/2805134206 shutdown_connections 2024-05-11T01:16:41.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.851+0000 7fa958b58640 1 -- 172.21.15.52:0/2805134206 wait complete. 2024-05-11T01:16:41.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.852+0000 7fa958b58640 1 Processor -- start 2024-05-11T01:16:41.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.852+0000 7fa958b58640 1 -- start start 2024-05-11T01:16:41.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa958b58640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9540d2d60 0x7fa9540ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa958b58640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d4e40 0x7fa9540cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa958b58640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95415f2f0 0x7fa95414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa958b58640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540d01e0 con 0x7fa9540d2d60 2024-05-11T01:16:41.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa958b58640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540d3720 con 0x7fa95415f2f0 2024-05-11T01:16:41.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa958b58640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa9540d0360 con 0x7fa9540d4e40 2024-05-11T01:16:41.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa95259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9540d2d60 0x7fa9540ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa95259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9540d2d60 0x7fa9540ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51444/0 (socket says 172.21.15.52:51444) 2024-05-11T01:16:41.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa95259c640 1 -- 172.21.15.52:0/554314378 learned_addr learned my addr 172.21.15.52:0/554314378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:41.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa951d9b640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d4e40 0x7fa9540cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa952d9d640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95415f2f0 0x7fa95414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa95259c640 1 -- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d4e40 msgr2=0x7fa9540cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa95259c640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d4e40 0x7fa9540cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa95259c640 1 -- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95415f2f0 msgr2=0x7fa95414fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:41.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.853+0000 7fa95259c640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95415f2f0 0x7fa95414fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:41.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.854+0000 7fa95259c640 1 -- 172.21.15.52:0/554314378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa954160360 con 0x7fa9540d2d60 2024-05-11T01:16:41.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.854+0000 7fa95259c640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9540d2d60 0x7fa9540ca6e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa944009d90 tx=0x7fa944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:41.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.854+0000 7fa94affd640 1 -- 172.21.15.52:0/554314378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944040070 con 0x7fa9540d2d60 2024-05-11T01:16:41.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.854+0000 7fa94affd640 1 -- 172.21.15.52:0/554314378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa944031c40 con 0x7fa9540d2d60 2024-05-11T01:16:41.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.854+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9540caf30 con 0x7fa9540d2d60 2024-05-11T01:16:41.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.854+0000 7fa94affd640 1 -- 172.21.15.52:0/554314378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944009450 con 0x7fa9540d2d60 2024-05-11T01:16:41.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.854+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9540cb4f0 con 0x7fa9540d2d60 2024-05-11T01:16:41.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.855+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9540695d0 con 0x7fa9540d2d60 2024-05-11T01:16:41.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.855+0000 7fa94affd640 1 -- 172.21.15.52:0/554314378 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fa944039040 con 0x7fa9540d2d60 2024-05-11T01:16:41.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.856+0000 7fa94affd640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9340762d0 0x7fa934078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:41.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.856+0000 7fa94affd640 1 -- 172.21.15.52:0/554314378 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7fa9440b8a20 con 0x7fa9540d2d60 2024-05-11T01:16:41.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.856+0000 7fa951d9b640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9340762d0 0x7fa934078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:41.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.857+0000 7fa951d9b640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9340762d0 0x7fa934078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa95415a8f0 tx=0x7fa93c007b10 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:41.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:41.860+0000 7fa94affd640 1 -- 172.21.15.52:0/554314378 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa944082110 con 0x7fa9540d2d60 2024-05-11T01:16:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.053+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fa9540d6080 con 0x7fa9540d2d60 2024-05-11T01:16:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.056+0000 7fa94affd640 1 -- 172.21.15.52:0/554314378 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v17)=0 v17) ==== 123+0+0 (secure 0 0 0) 0x7fa944081a10 con 0x7fa9540d2d60 2024-05-11T01:16:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.056+0000 7fa94affd640 1 -- 172.21.15.52:0/554314378 <== mon.0 v2:172.21.15.52:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa944081520 con 0x7fa9540d2d60 2024-05-11T01:16:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9340762d0 msgr2=0x7fa934078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9340762d0 0x7fa934078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa95415a8f0 tx=0x7fa93c007b10 comp rx=0 tx=0).stop 2024-05-11T01:16:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9540d2d60 msgr2=0x7fa9540ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9540d2d60 0x7fa9540ca6e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa944009d90 tx=0x7fa944004a40 comp rx=0 tx=0).stop 2024-05-11T01:16:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 shutdown_connections 2024-05-11T01:16:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9340762d0 0x7fa934078790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa95415f2f0 0x7fa95414fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9540d4e40 0x7fa9540cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 --2- 172.21.15.52:0/554314378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9540d2d60 0x7fa9540ca6e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 >> 172.21.15.52:0/554314378 conn(0x7fa95408ebb0 msgr2=0x7fa9540d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 shutdown_connections 2024-05-11T01:16:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.058+0000 7fa958b58640 1 -- 172.21.15.52:0/554314378 wait complete. 2024-05-11T01:16:42.068 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T01:16:42.068 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T01:16:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.173+0000 7fa98c0d1640 1 Processor -- start 2024-05-11T01:16:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.174+0000 7fa98c0d1640 1 -- start start 2024-05-11T01:16:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.174+0000 7fa98c0d1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa98415f430 0x7fa98415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.174+0000 7fa98c0d1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa9840d3d90 0x7fa9841606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.174+0000 7fa98c0d1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa984163250 con 0x7fa9841602b0 2024-05-11T01:16:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.174+0000 7fa98c0d1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 -- 0x7fa984163530 con 0x7fa9840d3d90 2024-05-11T01:16:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.174+0000 7fa98c0d1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa984163810 con 0x7fa98415f430 2024-05-11T01:16:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.175+0000 7fa98a66e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa9840d3d90 0x7fa9841606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.175+0000 7fa989e6d640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa98415f430 0x7fa98415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.175+0000 7fa98966c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa9841602b0 0x7fa9840d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.52:58716/0 (socket says 172.21.15.52:58716) 2024-05-11T01:16:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.175+0000 7fa98966c640 1 -- 172.21.15.52:0/3818413692 learned_addr learned my addr 172.21.15.52:0/3818413692 (peer_addr_for_me v1:172.21.15.52:0/0) 2024-05-11T01:16:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.175+0000 7fa98a66e640 1 -- 172.21.15.52:0/3818413692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa98415f430 msgr2=0x7fa98415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T01:16:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.175+0000 7fa98a66e640 1 --2- 172.21.15.52:0/3818413692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa98415f430 0x7fa98415fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.175+0000 7fa98a66e640 1 -- 172.21.15.52:0/3818413692 >> v1:172.21.15.119:6789/0 conn(0x7fa9841602b0 legacy=0x7fa9840d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.175+0000 7fa988e6b640 1 -- 172.21.15.52:0/3818413692 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4060457089 0 0) 0x7fa984163250 con 0x7fa9841602b0 2024-05-11T01:16:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.175+0000 7fa98a66e640 1 -- 172.21.15.52:0/3818413692 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa984163af0 con 0x7fa9840d3d90 2024-05-11T01:16:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.176+0000 7fa98a66e640 1 --2- 172.21.15.52:0/3818413692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840d3d90 0x7fa9841606f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa974009d90 tx=0x7fa97402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b278da31ee996cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.176+0000 7fa988e6b640 1 -- 172.21.15.52:0/3818413692 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974007740 con 0x7fa9840d3d90 2024-05-11T01:16:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.176+0000 7fa988e6b640 1 -- 172.21.15.52:0/3818413692 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa9740078e0 con 0x7fa9840d3d90 2024-05-11T01:16:42.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.176+0000 7fa988e6b640 1 -- 172.21.15.52:0/3818413692 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974007cd0 con 0x7fa9840d3d90 2024-05-11T01:16:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.177+0000 7fa98c0d1640 1 -- 172.21.15.52:0/3818413692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840d3d90 msgr2=0x7fa9841606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.177+0000 7fa98c0d1640 1 --2- 172.21.15.52:0/3818413692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840d3d90 0x7fa9841606f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa974009d90 tx=0x7fa97402a660 comp rx=0 tx=0).stop 2024-05-11T01:16:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.177+0000 7fa98c0d1640 1 -- 172.21.15.52:0/3818413692 shutdown_connections 2024-05-11T01:16:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.177+0000 7fa98c0d1640 1 --2- 172.21.15.52:0/3818413692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840d3d90 0x7fa9841606f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.177+0000 7fa98c0d1640 1 --2- 172.21.15.52:0/3818413692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa98415f430 0x7fa98415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.177+0000 7fa98c0d1640 1 -- 172.21.15.52:0/3818413692 >> 172.21.15.52:0/3818413692 conn(0x7fa98408eb70 msgr2=0x7fa98409e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.177+0000 7fa98c0d1640 1 -- 172.21.15.52:0/3818413692 shutdown_connections 2024-05-11T01:16:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.177+0000 7fa98c0d1640 1 -- 172.21.15.52:0/3818413692 wait complete. 2024-05-11T01:16:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.178+0000 7fa98c0d1640 1 Processor -- start 2024-05-11T01:16:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.178+0000 7fa98c0d1640 1 -- start start 2024-05-11T01:16:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98c0d1640 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9840d3d90 0x7fa98415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98c0d1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98415f430 0x7fa98414aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98c0d1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9841602b0 0x7fa9841581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98c0d1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa984163030 con 0x7fa9841602b0 2024-05-11T01:16:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98c0d1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa984162eb0 con 0x7fa98415f430 2024-05-11T01:16:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98c0d1640 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 -- 0x7fa9840d2260 con 0x7fa9840d3d90 2024-05-11T01:16:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98a66e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9841602b0 0x7fa9841581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98a66e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9841602b0 0x7fa9841581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51452/0 (socket says 172.21.15.52:51452) 2024-05-11T01:16:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98a66e640 1 -- 172.21.15.52:0/4140136333 learned_addr learned my addr 172.21.15.52:0/4140136333 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-11T01:16:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98966c640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98415f430 0x7fa98414aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa989e6d640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9840d3d90 0x7fa98415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98a66e640 1 -- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9840d3d90 msgr2=0x7fa98415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.179+0000 7fa98a66e640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9840d3d90 0x7fa98415cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa98a66e640 1 -- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98415f430 msgr2=0x7fa98414aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa98a66e640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98415f430 0x7fa98414aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa98a66e640 1 -- 172.21.15.52:0/4140136333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98415d8c0 con 0x7fa9841602b0 2024-05-11T01:16:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa98966c640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98415f430 0x7fa98414aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T01:16:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa98a66e640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9841602b0 0x7fa9841581c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa974009d90 tx=0x7fa974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa9827fc640 1 -- 172.21.15.52:0/4140136333 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974041070 con 0x7fa9841602b0 2024-05-11T01:16:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa9827fc640 1 -- 172.21.15.52:0/4140136333 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa974031c40 con 0x7fa9841602b0 2024-05-11T01:16:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98415dba0 con 0x7fa9841602b0 2024-05-11T01:16:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa9827fc640 1 -- 172.21.15.52:0/4140136333 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974009450 con 0x7fa9841602b0 2024-05-11T01:16:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.180+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98415d490 con 0x7fa9841602b0 2024-05-11T01:16:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.181+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa984069780 con 0x7fa9841602b0 2024-05-11T01:16:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.182+0000 7fa9827fc640 1 -- 172.21.15.52:0/4140136333 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 98201+0+0 (secure 0 0 0) 0x7fa974039040 con 0x7fa9841602b0 2024-05-11T01:16:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.182+0000 7fa9827fc640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9700762d0 0x7fa970078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T01:16:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.182+0000 7fa9827fc640 1 -- 172.21.15.52:0/4140136333 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6323+0+0 (secure 0 0 0) 0x7fa9740b8ab0 con 0x7fa9841602b0 2024-05-11T01:16:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.182+0000 7fa989e6d640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9700762d0 0x7fa970078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T01:16:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.183+0000 7fa989e6d640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9700762d0 0x7fa970078790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009b40 tx=0x7fa97c009190 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T01:16:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.186+0000 7fa9827fc640 1 -- 172.21.15.52:0/4140136333 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa974081ac0 con 0x7fa9841602b0 2024-05-11T01:16:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.417+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9840a2e40 con 0x7fa9841602b0 2024-05-11T01:16:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.417+0000 7fa9827fc640 1 -- 172.21.15.52:0/4140136333 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5178 (secure 0 0 0) 0x7fa974085dc0 con 0x7fa9841602b0 2024-05-11T01:16:42.417 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-11T01:16:42.417 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5165,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/4021589378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4021589378},{"type":"v1","addr":"172.21.15.119:6836","nonce":4021589378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/456790373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":456790373},{"type":"v1","addr":"172.21.15.119:6837","nonce":456790373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5646,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/1460397063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1460397063},{"type":"v1","addr":"172.21.15.52:6834","nonce":1460397063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T01:13:58.799356+0000","modified":"2024-05-11T01:14:01.810259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/59064433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":59064433},{"type":"v1","addr":"172.21.15.52:6835","nonce":59064433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5649,"qdb_cluster":[5649]},"id":3}]} 2024-05-11T01:16:42.417 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-05-11T01:16:42.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9700762d0 msgr2=0x7fa970078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:42.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9700762d0 0x7fa970078790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009b40 tx=0x7fa97c009190 comp rx=0 tx=0).stop 2024-05-11T01:16:42.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9841602b0 msgr2=0x7fa9841581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T01:16:42.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9841602b0 0x7fa9841581c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa974009d90 tx=0x7fa974004a40 comp rx=0 tx=0).stop 2024-05-11T01:16:42.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 shutdown_connections 2024-05-11T01:16:42.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:6832/4040438696,v1:172.21.15.119:6833/4040438696] conn(0x7fa9700762d0 0x7fa970078790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9841602b0 0x7fa9841581c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98415f430 0x7fa98414aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 --2- 172.21.15.52:0/4140136333 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9840d3d90 0x7fa98415cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T01:16:42.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 >> 172.21.15.52:0/4140136333 conn(0x7fa98408eb70 msgr2=0x7fa9840d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T01:16:42.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 shutdown_connections 2024-05-11T01:16:42.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-11T01:16:42.419+0000 7fa98c0d1640 1 -- 172.21.15.52:0/4140136333 wait complete. 2024-05-11T01:16:42.429 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-11T01:16:42.429 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-11T01:16:42.429 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T01:16:42.430 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.439 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T01:16:42.439 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T01:16:42.439 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.489 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T01:16:42.489 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T01:16:42.489 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.502 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T01:16:42.502 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T01:16:42.502 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.553 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T01:16:42.553 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-11T01:16:42.554 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-11T01:16:42.554 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.581 INFO:tasks.ceph.osd.0:Stopped 2024-05-11T01:16:42.581 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-11T01:16:42.581 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.657 INFO:tasks.ceph.osd.1:Stopped 2024-05-11T01:16:42.657 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-11T01:16:42.657 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.723 INFO:tasks.ceph.osd.2:Stopped 2024-05-11T01:16:42.724 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-11T01:16:42.724 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.794 INFO:tasks.ceph.osd.3:Stopped 2024-05-11T01:16:42.795 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-11T01:16:42.795 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.826 INFO:tasks.ceph.osd.4:Stopped 2024-05-11T01:16:42.826 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-11T01:16:42.827 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.897 INFO:tasks.ceph.osd.5:Stopped 2024-05-11T01:16:42.898 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-11T01:16:42.898 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:42.970 INFO:tasks.ceph.osd.6:Stopped 2024-05-11T01:16:42.971 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-11T01:16:42.971 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:43.037 INFO:tasks.ceph.osd.7:Stopped 2024-05-11T01:16:43.037 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-11T01:16:43.037 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-11T01:16:43.037 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:43.065 INFO:tasks.ceph.mgr.y:Stopped 2024-05-11T01:16:43.065 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-11T01:16:43.065 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:43.107 INFO:tasks.ceph.mgr.x:Stopped 2024-05-11T01:16:43.107 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-11T01:16:43.108 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-11T01:16:43.108 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:43.119 INFO:tasks.ceph.mon.a:Stopped 2024-05-11T01:16:43.119 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-11T01:16:43.120 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:43.160 INFO:tasks.ceph.mon.b:Stopped 2024-05-11T01:16:43.161 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-11T01:16:43.161 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T01:16:43.213 INFO:tasks.ceph.mon.c:Stopped 2024-05-11T01:16:43.213 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-11T01:16:43.213 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-05-11T01:16:43.247 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:16:43.247 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-11T01:16:43.492 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:16:43.492 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-11T01:16:43.707 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:16:43.707 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-11T01:16:43.757 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:16:43.757 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-11T01:16:43.840 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:16:43.841 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-11T01:16:43.982 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:16:43.982 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-11T01:16:44.069 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:16:44.069 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-11T01:16:44.125 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:16:44.126 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-11T01:16:44.184 INFO:tasks.ceph:Archiving mon data... 2024-05-11T01:16:44.186 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/data/mon.a.tgz 2024-05-11T01:16:44.186 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-05-11T01:16:44.202 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.ElhULBBjzn 2024-05-11T01:16:44.203 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ElhULBBjzn 2024-05-11T01:16:44.387 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.ElhULBBjzn 2024-05-11T01:16:44.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:16:44.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:16:44.514 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.ElhULBBjzn is 620KB 2024-05-11T01:16:44.592 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.ElhULBBjzn 2024-05-11T01:16:44.608 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/data/mon.b.tgz 2024-05-11T01:16:44.608 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-05-11T01:16:44.624 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.nbUlnD3QDN 2024-05-11T01:16:44.625 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nbUlnD3QDN 2024-05-11T01:16:44.805 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.nbUlnD3QDN 2024-05-11T01:16:44.888 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.nbUlnD3QDN is 630KB 2024-05-11T01:16:44.989 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.nbUlnD3QDN 2024-05-11T01:16:45.004 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/data/mon.c.tgz 2024-05-11T01:16:45.004 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-05-11T01:16:45.058 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.jx2mSBN0Am 2024-05-11T01:16:45.059 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.jx2mSBN0Am 2024-05-11T01:16:45.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.jx2mSBN0Am 2024-05-11T01:16:45.311 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.jx2mSBN0Am is 630KB 2024-05-11T01:16:45.413 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.jx2mSBN0Am 2024-05-11T01:16:45.428 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-11T01:16:45.428 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-11T01:16:45.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-11T01:16:45.515 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-11T01:16:45.547 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-11T01:16:45.547 INFO:tasks.ceph:Archiving crash dumps... 2024-05-11T01:16:45.549 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/remote/smithi052/crash 2024-05-11T01:16:45.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-11T01:16:45.584 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/remote/smithi119/crash 2024-05-11T01:16:45.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-11T01:16:45.610 INFO:tasks.ceph:Shutting down logrotate 2024-05-11T01:16:45.610 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:16:45.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T01:16:45.714 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-11T01:16:45.742 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-11T01:16:45.767 INFO:tasks.ceph:Compressing logs... 2024-05-11T01:16:45.767 DEBUG:teuthology.orchestra.run.smithi052:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T01:16:45.785 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T01:16:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43208.log 2024-05-11T01:16:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42578.log 2024-05-11T01:16:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.43208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47871.log 2024-05-11T01:16:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55842.log 2024-05-11T01:16:45.816 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42578.log.gz 2024-05-11T01:16:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47871.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.43208.log.gz 2024-05-11T01:16:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48031.log 2024-05-11T01:16:45.817 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47871.log.gz 2024-05-11T01:16:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51001.log 2024-05-11T01:16:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.48031.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.55842.log.gz 2024-05-11T01:16:45.817 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.43749.log 2024-05-11T01:16:45.817 INFO:teuthology.orchestra.run.smithi052.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48031.log.gz 2024-05-11T01:16:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51001.log: /var/log/ceph/ceph-client.admin.45903.log 2024-05-11T01:16:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.0.43749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53512.log 2024-05-11T01:16:45.817 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51001.log.gz 2024-05-11T01:16:45.818 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47516.log 2024-05-11T01:16:45.818 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45903.log.gz 2024-05-11T01:16:45.818 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53512.log: 89.3% -- replaced with /var/log/ceph/ceph-client.0.43749.log.gz 2024-05-11T01:16:45.818 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54034.log 2024-05-11T01:16:45.818 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53512.log.gz 2024-05-11T01:16:45.818 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47446.log 2024-05-11T01:16:45.818 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47516.log.gz/var/log/ceph/ceph-client.admin.54034.log: 2024-05-11T01:16:45.818 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41510.log 2024-05-11T01:16:45.818 INFO:teuthology.orchestra.run.smithi052.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54034.log.gz/var/log/ceph/ceph-client.admin.47446.log: 2024-05-11T01:16:45.819 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52887.log 2024-05-11T01:16:45.819 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47446.log.gz 2024-05-11T01:16:45.819 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42376.log 2024-05-11T01:16:45.819 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.52887.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41510.log.gz 2024-05-11T01:16:45.819 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47220.log 2024-05-11T01:16:45.819 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52887.log.gz 2024-05-11T01:16:45.819 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44987.log 2024-05-11T01:16:45.819 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9%/var/log/ceph/ceph-client.admin.47220.log: -- replaced with /var/log/ceph/ceph-client.admin.42376.log.gz 2024-05-11T01:16:45.819 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47262.log 2024-05-11T01:16:45.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44987.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47220.log.gz 2024-05-11T01:16:45.820 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53983.log 2024-05-11T01:16:45.820 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44987.log.gz 2024-05-11T01:16:45.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54784.log 2024-05-11T01:16:45.820 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47262.log.gz 2024-05-11T01:16:45.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46527.log 2024-05-11T01:16:45.820 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53983.log.gz 2024-05-11T01:16:45.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.0.54784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53895.log 2024-05-11T01:16:45.820 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46737.log 2024-05-11T01:16:45.821 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46527.log.gz 2024-05-11T01:16:45.821 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5/var/log/ceph/ceph-client.admin.53895.log: --verbose -- /var/log/ceph/ceph-client.admin.45061.log 2024-05-11T01:16:45.841 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35946.log 2024-05-11T01:16:45.841 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-11T01:16:45.841 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42615.log 2024-05-11T01:16:45.841 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35946.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-11T01:16:45.841 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35946.log.gz 2024-05-11T01:16:45.841 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.3.42615.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-11T01:16:45.841 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-11T01:16:45.842 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.3.42615.log.gz 2024-05-11T01:16:45.842 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-11T01:16:45.842 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-11T01:16:45.842 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-11T01:16:45.842 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-11T01:16:45.842 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-11T01:16:45.842 INFO:teuthology.orchestra.run.smithi119.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-11T01:16:45.842 INFO:teuthology.orchestra.run.smithi119.stderr: 93.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-11T01:16:45.853 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35987.log 2024-05-11T01:16:45.853 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36108.log 2024-05-11T01:16:45.853 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35987.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35987.log.gz 2024-05-11T01:16:45.853 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35896.log 2024-05-11T01:16:45.853 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36108.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36108.log.gz 2024-05-11T01:16:45.853 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36068.log 2024-05-11T01:16:45.853 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35896.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35896.log.gz 2024-05-11T01:16:45.854 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35816.log 2024-05-11T01:16:45.854 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36068.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36068.log.gz 2024-05-11T01:16:45.854 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36028.log 2024-05-11T01:16:45.854 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35816.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35816.log.gz 2024-05-11T01:16:45.854 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35856.log 2024-05-11T01:16:45.854 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36028.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36028.log.gz 2024-05-11T01:16:45.854 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42159.log 2024-05-11T01:16:45.855 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.35856.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35856.log.gz 2024-05-11T01:16:45.855 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-11T01:16:45.861 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.1.42159.log: 87.8% -- replaced with /var/log/ceph/ceph-client.1.42159.log.gz 2024-05-11T01:16:45.861 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-11T01:16:45.861 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-11T01:16:45.861 INFO:teuthology.orchestra.run.smithi119.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-11T01:16:45.897 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46737.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53895.log.gz 2024-05-11T01:16:45.897 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47575.log 2024-05-11T01:16:45.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45061.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-11T01:16:45.898 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45061.log.gz 2024-05-11T01:16:45.898 INFO:teuthology.orchestra.run.smithi052.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46737.log.gz 2024-05-11T01:16:45.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42036.log 2024-05-11T01:16:45.898 INFO:teuthology.orchestra.run.smithi052.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47575.log.gz 2024-05-11T01:16:45.898 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45364.log 2024-05-11T01:16:45.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph.log: 95.2% -- replaced with /var/log/ceph/ceph-client.0.54784.log.gz 2024-05-11T01:16:45.898 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44504.log 2024-05-11T01:16:45.898 INFO:teuthology.orchestra.run.smithi052.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42036.log.gz 2024-05-11T01:16:45.899 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42371.log 2024-05-11T01:16:45.899 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45364.log.gz 2024-05-11T01:16:45.899 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55674.log 2024-05-11T01:16:45.899 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42371.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.44504.log.gz 2024-05-11T01:16:45.899 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44867.log 2024-05-11T01:16:45.899 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42371.log.gz 2024-05-11T01:16:45.899 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54416.log 2024-05-11T01:16:45.899 INFO:teuthology.orchestra.run.smithi052.stderr: 87.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-11T01:16:45.899 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55674.log.gz 2024-05-11T01:16:45.900 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46443.log 2024-05-11T01:16:45.900 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54416.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44867.log.gz 2024-05-11T01:16:45.900 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-11T01:16:45.900 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54416.log.gz 2024-05-11T01:16:45.900 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40907.log 2024-05-11T01:16:45.900 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46443.log.gz 2024-05-11T01:16:45.900 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49384.log 2024-05-11T01:16:45.900 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.40907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41071.log 2024-05-11T01:16:45.900 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40907.log.gz/var/log/ceph/ceph-client.admin.49384.log: 2024-05-11T01:16:45.901 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56052.log 2024-05-11T01:16:45.901 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49384.log.gz 2024-05-11T01:16:45.901 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54941.log 2024-05-11T01:16:45.901 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41071.log.gz 2024-05-11T01:16:45.901 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.56052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47753.log 2024-05-11T01:16:45.901 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54941.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56052.log.gz 2024-05-11T01:16:45.901 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55086.log 2024-05-11T01:16:45.901 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54941.log.gz 2024-05-11T01:16:45.902 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46397.log 2024-05-11T01:16:45.902 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47753.log.gz 2024-05-11T01:16:45.902 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5/var/log/ceph/ceph-client.admin.55086.log: --verbose -- /var/log/ceph/ceph-client.admin.52698.log 2024-05-11T01:16:45.902 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41300.log 2024-05-11T01:16:45.902 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55086.log.gz 2024-05-11T01:16:45.902 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46397.log.gz 2024-05-11T01:16:45.902 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.52698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42044.log 2024-05-11T01:16:45.902 INFO:teuthology.orchestra.run.smithi052.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52698.log.gz 2024-05-11T01:16:45.902 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54290.log 2024-05-11T01:16:45.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42044.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41300.log.gz 2024-05-11T01:16:45.903 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53031.log 2024-05-11T01:16:45.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54290.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42044.log.gz 2024-05-11T01:16:45.903 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45574.log 2024-05-11T01:16:45.903 INFO:teuthology.orchestra.run.smithi052.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54290.log.gz 2024-05-11T01:16:45.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47693.log 2024-05-11T01:16:45.903 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53031.log.gz 2024-05-11T01:16:45.903 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53073.log 2024-05-11T01:16:45.904 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45574.log.gz 2024-05-11T01:16:45.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55800.log 2024-05-11T01:16:45.904 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47693.log.gz 2024-05-11T01:16:45.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56261.log 2024-05-11T01:16:45.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55800.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53073.log.gz 2024-05-11T01:16:45.904 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-11T01:16:45.904 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55800.log.gz 2024-05-11T01:16:45.904 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.56261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52316.log 2024-05-11T01:16:45.904 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56261.log.gz 2024-05-11T01:16:45.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52053.log 2024-05-11T01:16:45.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.52316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53176.log 2024-05-11T01:16:45.905 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52316.log.gz 2024-05-11T01:16:45.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.52053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45196.log 2024-05-11T01:16:45.905 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49182.log 2024-05-11T01:16:45.905 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52053.log.gz 2024-05-11T01:16:45.905 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53176.log: /var/log/ceph/ceph-client.admin.45196.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-11T01:16:45.905 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53176.log.gz 2024-05-11T01:16:45.906 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53939.log 2024-05-11T01:16:45.906 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45196.log.gz 2024-05-11T01:16:45.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49561.log 2024-05-11T01:16:45.906 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49182.log.gz 2024-05-11T01:16:45.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42380.log 2024-05-11T01:16:45.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49561.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53939.log.gz 2024-05-11T01:16:45.906 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47370.log 2024-05-11T01:16:45.906 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49561.log.gz 2024-05-11T01:16:45.906 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45616.log 2024-05-11T01:16:45.907 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42380.log.gz 2024-05-11T01:16:45.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52989.log 2024-05-11T01:16:45.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45616.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47370.log.gz 2024-05-11T01:16:45.907 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2024-05-11T01:16:45.907 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45616.log.gz 2024-05-11T01:16:45.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.52989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55144.log 2024-05-11T01:16:45.907 INFO:teuthology.orchestra.run.smithi052.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52989.log.gz 2024-05-11T01:16:45.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.48283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42718.log 2024-05-11T01:16:45.907 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55144.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2024-05-11T01:16:45.908 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41636.log 2024-05-11T01:16:45.908 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55144.log.gz 2024-05-11T01:16:45.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49098.log 2024-05-11T01:16:45.908 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42718.log.gz 2024-05-11T01:16:45.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41636.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-11T01:16:45.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49098.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41636.log.gz 2024-05-11T01:16:45.908 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46132.log 2024-05-11T01:16:45.908 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49098.log.gz 2024-05-11T01:16:45.908 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55338.log 2024-05-11T01:16:45.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41822.log 2024-05-11T01:16:45.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55338.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46132.log.gz 2024-05-11T01:16:45.909 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41678.log 2024-05-11T01:16:45.909 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42037.log 2024-05-11T01:16:45.909 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55338.log.gz 2024-05-11T01:16:45.909 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41822.log: /var/log/ceph/ceph-client.admin.41678.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41822.log.gz 2024-05-11T01:16:45.909 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54164.log 2024-05-11T01:16:45.909 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41678.log.gz 2024-05-11T01:16:45.909 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47930.log 2024-05-11T01:16:45.910 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42037.log: /var/log/ceph/ceph-client.admin.54164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49856.log 2024-05-11T01:16:45.910 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42037.log.gz 2024-05-11T01:16:45.910 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54164.log.gz 2024-05-11T01:16:45.910 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55758.log 2024-05-11T01:16:45.910 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47930.log.gz 2024-05-11T01:16:45.910 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49502.log 2024-05-11T01:16:45.910 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49856.log.gz 2024-05-11T01:16:45.910 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49797.log 2024-05-11T01:16:45.910 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55758.log: /var/log/ceph/ceph-client.admin.49502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53344.log 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55758.log.gz 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49502.log.gz 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47063.log 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45280.log 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49797.log.gz 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53344.log: /var/log/ceph/ceph-client.0.47063.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53344.log.gz 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42375.log 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45280.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46090.log 2024-05-11T01:16:45.911 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45280.log.gz 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41384.log 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53428.log 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42375.log.gz 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47812.log 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46090.log.gz 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51790.log 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41384.log.gz 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53428.log.gz 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44394.log 2024-05-11T01:16:45.912 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47812.log: /var/log/ceph/ceph-client.admin.51790.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47812.log.gz 2024-05-11T01:16:45.913 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41342.log 2024-05-11T01:16:45.913 INFO:teuthology.orchestra.run.smithi052.stderr: 87.7%/var/log/ceph/ceph-client.admin.44394.log: -- replaced with /var/log/ceph/ceph-client.admin.51790.log.gz 2024-05-11T01:16:45.913 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53853.log 2024-05-11T01:16:45.913 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44394.log.gz 2024-05-11T01:16:45.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41342.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-11T01:16:45.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53853.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41342.log.gz 2024-05-11T01:16:45.913 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41029.log 2024-05-11T01:16:45.913 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53853.log.gz 2024-05-11T01:16:45.913 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.1.log: 93.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-11T01:16:45.914 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-11T01:16:45.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55926.log 2024-05-11T01:16:45.914 INFO:teuthology.orchestra.run.smithi052.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41029.log.gz 2024-05-11T01:16:45.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55590.log 2024-05-11T01:16:45.914 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55926.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-11T01:16:45.914 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9%/var/log/ceph/ceph-client.admin.55590.log: -- replaced with /var/log/ceph/ceph-client.admin.55926.log.gz 2024-05-11T01:16:45.914 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56136.log 2024-05-11T01:16:45.914 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55590.log.gz 2024-05-11T01:16:45.914 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29351.log 2024-05-11T01:16:45.915 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.56136.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56136.log.gz 2024-05-11T01:16:45.915 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49443.log 2024-05-11T01:16:45.915 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph.tmp-client.admin.29351.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29351.log.gz 2024-05-11T01:16:45.915 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45238.log 2024-05-11T01:16:45.915 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50178.log 2024-05-11T01:16:45.915 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45238.log: /var/log/ceph/ceph-client.admin.49443.log: 88.3% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49443.log.gz 2024-05-11T01:16:45.915 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45238.log.gz 2024-05-11T01:16:45.915 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53133.log 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.50178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42717.log 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50178.log.gz 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55380.log 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42717.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53133.log.gz 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53386.log 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42717.log.gz 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48972.log 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55884.log 2024-05-11T01:16:45.916 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55380.log.gz 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53386.log.gz 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.48972.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48972.log.gz 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42914.log 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41468.log 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr: 87.7%/var/log/ceph/ceph-client.admin.42914.log: -- replaced with /var/log/ceph/ceph-client.admin.55884.log.gz 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43310.log 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42914.log.gz 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49325.log 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41468.log: /var/log/ceph/ceph-client.admin.43310.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43310.log.gz 2024-05-11T01:16:45.917 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41468.log.gz 2024-05-11T01:16:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48409.log 2024-05-11T01:16:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55716.log 2024-05-11T01:16:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.48409.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49325.log.gz 2024-05-11T01:16:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42046.log 2024-05-11T01:16:45.918 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48409.log.gz 2024-05-11T01:16:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53218.log 2024-05-11T01:16:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55716.log: /var/log/ceph/ceph-client.admin.42046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42714.log 2024-05-11T01:16:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53218.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55716.log.gz 2024-05-11T01:16:45.919 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42046.log.gz 2024-05-11T01:16:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50738.log 2024-05-11T01:16:45.919 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53218.log.gz 2024-05-11T01:16:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42714.log: /var/log/ceph/ceph-client.admin.54206.log 2024-05-11T01:16:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.50738.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47315.log 2024-05-11T01:16:45.919 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42714.log.gz 2024-05-11T01:16:45.919 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50738.log.gz 2024-05-11T01:16:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35380.log 2024-05-11T01:16:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44714.log 2024-05-11T01:16:45.920 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54206.log.gz 2024-05-11T01:16:45.920 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47315.log.gz/var/log/ceph/ceph-client.admin.35380.log: 2024-05-11T01:16:45.920 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53811.log 2024-05-11T01:16:45.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46048.log 2024-05-11T01:16:45.920 INFO:teuthology.orchestra.run.smithi052.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35380.log.gz 2024-05-11T01:16:45.920 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53811.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44714.log.gz 2024-05-11T01:16:45.920 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49140.log 2024-05-11T01:16:45.920 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53811.log.gz 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46048.log: -- /var/log/ceph/ceph-client.admin.48325.log 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49140.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46048.log.gz 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45322.log 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49140.log.gz 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48199.log 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.48325.log: /var/log/ceph/ceph-client.admin.45322.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0%/var/log/ceph/ceph-client.admin.48199.log: -- replaced with /var/log/ceph/ceph-client.admin.48325.log.gz 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45322.log.gz 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52845.log 2024-05-11T01:16:45.921 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48199.log.gz 2024-05-11T01:16:45.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48115.log 2024-05-11T01:16:45.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.52845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52579.log 2024-05-11T01:16:45.922 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7%/var/log/ceph/ceph-client.admin.48115.log: -- replaced with /var/log/ceph/ceph-client.admin.52845.log.gz 2024-05-11T01:16:45.922 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48115.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49266.log 2024-05-11T01:16:45.922 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-11T01:16:45.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.52579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46174.log 2024-05-11T01:16:45.922 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52579.log.gz 2024-05-11T01:16:45.922 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48241.log 2024-05-11T01:16:45.923 INFO:teuthology.orchestra.run.smithi052.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49266.log.gz 2024-05-11T01:16:45.923 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42382.log 2024-05-11T01:16:45.923 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.48241.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-05-11T01:16:45.923 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48241.log.gz 2024-05-11T01:16:45.923 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41864.log 2024-05-11T01:16:45.923 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42382.log: -- /var/log/ceph/ceph-client.admin.45738.log 2024-05-11T01:16:45.923 INFO:teuthology.orchestra.run.smithi052.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42382.log.gz 2024-05-11T01:16:45.923 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42381.log 2024-05-11T01:16:45.923 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45738.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41864.log.gz 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45738.log.gz 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51527.log 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46355.log 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.51527.log: /var/log/ceph/ceph-client.admin.42381.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42381.log.gz 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51527.log.gz 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47989.log 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48367.log 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46355.log.gz 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54122.log 2024-05-11T01:16:45.924 INFO:teuthology.orchestra.run.smithi052.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47989.log.gz 2024-05-11T01:16:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.48367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48888.log 2024-05-11T01:16:45.925 INFO:teuthology.orchestra.run.smithi052.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48367.log.gz 2024-05-11T01:16:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45946.log 2024-05-11T01:16:45.925 INFO:teuthology.orchestra.run.smithi052.stderr: 88.1%/var/log/ceph/ceph-client.admin.48888.log: -- replaced with /var/log/ceph/ceph-client.admin.54122.log.gz 2024-05-11T01:16:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48888.log.gz 2024-05-11T01:16:45.925 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42034.log 2024-05-11T01:16:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44809.log 2024-05-11T01:16:45.925 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8%/var/log/ceph/ceph-client.admin.42034.log: -- replaced with /var/log/ceph/ceph-client.admin.45946.log.gz 2024-05-11T01:16:45.925 INFO:teuthology.orchestra.run.smithi052.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40809.log 2024-05-11T01:16:45.926 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42034.log.gz 2024-05-11T01:16:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44809.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53649.log 2024-05-11T01:16:45.926 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44809.log.gz 2024-05-11T01:16:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.40809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46653.log 2024-05-11T01:16:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53649.log: 89.1% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40809.log.gz 2024-05-11T01:16:45.926 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53649.log.gz 2024-05-11T01:16:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46262.log 2024-05-11T01:16:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42715.log 2024-05-11T01:16:45.926 INFO:teuthology.orchestra.run.smithi052.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46653.log.gz 2024-05-11T01:16:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41201.log 2024-05-11T01:16:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42715.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46262.log.gz 2024-05-11T01:16:45.927 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42715.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56010.log 2024-05-11T01:16:45.927 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-11T01:16:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42713.log 2024-05-11T01:16:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.56010.log: /var/log/ceph/ceph-client.admin.41201.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41201.log.gz 2024-05-11T01:16:45.927 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56010.log.gz 2024-05-11T01:16:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52792.log 2024-05-11T01:16:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56221.log 2024-05-11T01:16:45.928 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42713.log.gz 2024-05-11T01:16:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.52792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56178.log 2024-05-11T01:16:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.56221.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52792.log.gz 2024-05-11T01:16:45.928 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56221.log.gz 2024-05-11T01:16:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41250.log 2024-05-11T01:16:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54374.log 2024-05-11T01:16:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.56178.log: /var/log/ceph/ceph-client.admin.41250.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56178.log.gz 2024-05-11T01:16:45.928 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41250.log.gz 2024-05-11T01:16:45.929 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2024-05-11T01:16:45.929 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55422.log 2024-05-11T01:16:45.929 INFO:teuthology.orchestra.run.smithi052.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54374.log.gz 2024-05-11T01:16:45.929 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54458.log 2024-05-11T01:16:45.929 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55422.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gz 2024-05-11T01:16:45.984 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55464.log 2024-05-11T01:16:45.984 INFO:teuthology.orchestra.run.smithi052.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55422.log.gz 2024-05-11T01:16:45.984 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53302.log 2024-05-11T01:16:45.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55464.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55464.log.gz 2024-05-11T01:16:45.985 INFO:teuthology.orchestra.run.smithi052.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54458.log.gz 2024-05-11T01:16:45.985 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46313.log 2024-05-11T01:16:45.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49898.log 2024-05-11T01:16:45.985 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53302.log.gz 2024-05-11T01:16:45.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53260.log 2024-05-11T01:16:45.985 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49898.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46313.log.gz 2024-05-11T01:16:45.985 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49014.log 2024-05-11T01:16:45.986 INFO:teuthology.orchestra.run.smithi052.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49898.log.gz 2024-05-11T01:16:45.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48930.log 2024-05-11T01:16:45.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49014.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53260.log.gz 2024-05-11T01:16:45.986 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49014.log.gzgzip 2024-05-11T01:16:45.986 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41552.log 2024-05-11T01:16:45.986 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45819.log 2024-05-11T01:16:45.986 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41552.log: /var/log/ceph/ceph-client.admin.48930.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48930.log.gz 2024-05-11T01:16:45.986 INFO:teuthology.orchestra.run.smithi052.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41552.log.gz 2024-05-11T01:16:45.986 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45532.log 2024-05-11T01:16:45.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49056.log 2024-05-11T01:16:45.987 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45819.log.gz 2024-05-11T01:16:45.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-05-11T01:16:45.987 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49056.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45532.log.gz 2024-05-11T01:16:45.987 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49056.log.gz 2024-05-11T01:16:45.987 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41113.log 2024-05-11T01:16:45.987 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5/var/log/ceph/ceph-client.admin.43124.log: --verbose -- /var/log/ceph/ceph-client.admin.49224.log 2024-05-11T01:16:45.987 INFO:teuthology.orchestra.run.smithi052.stderr: 89.4%/var/log/ceph/ceph-client.admin.41113.log: -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-05-11T01:16:45.988 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41113.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54248.log 2024-05-11T01:16:45.988 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-11T01:16:45.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42379.log 2024-05-11T01:16:45.988 INFO:teuthology.orchestra.run.smithi052.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49224.log.gz 2024-05-11T01:16:45.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55506.log 2024-05-11T01:16:45.988 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42379.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54248.log.gz 2024-05-11T01:16:45.988 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42379.log.gzgzip 2024-05-11T01:16:45.988 INFO:teuthology.orchestra.run.smithi052.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53554.log 2024-05-11T01:16:45.988 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44588.log 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55506.log: /var/log/ceph/ceph-client.admin.53554.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53554.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55506.log.gz 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45988.log 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46803.log 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44588.log.gz 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50458.log 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46803.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45988.log.gz 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44352.log 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46803.log.gz 2024-05-11T01:16:45.989 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-05-11T01:16:45.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.50458.log: /var/log/ceph/ceph-client.admin.44352.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50458.log.gz 2024-05-11T01:16:45.990 INFO:teuthology.orchestra.run.smithi052.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44352.log.gz 2024-05-11T01:16:45.990 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35644.log 2024-05-11T01:16:45.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49620.log 2024-05-11T01:16:45.990 INFO:teuthology.orchestra.run.smithi052.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-05-11T01:16:45.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54332.log 2024-05-11T01:16:45.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49620.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35644.log.gz 2024-05-11T01:16:45.990 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46695.log 2024-05-11T01:16:45.990 INFO:teuthology.orchestra.run.smithi052.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49620.log.gz 2024-05-11T01:16:45.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44546.log 2024-05-11T01:16:45.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46695.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54332.log.gz 2024-05-11T01:16:45.991 INFO:teuthology.orchestra.run.smithi052.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46695.log.gz 2024-05-11T01:16:45.991 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52947.log 2024-05-11T01:16:45.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42043.log 2024-05-11T01:16:45.991 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44546.log.gz 2024-05-11T01:16:45.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.52947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55548.log 2024-05-11T01:16:45.991 INFO:teuthology.orchestra.run.smithi052.stderr: 88.7%/var/log/ceph/ceph-client.admin.42043.log: -- replaced with /var/log/ceph/ceph-client.admin.52947.log.gz 2024-05-11T01:16:45.991 INFO:teuthology.orchestra.run.smithi052.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48157.log 2024-05-11T01:16:45.992 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42043.log.gz 2024-05-11T01:16:45.992 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45490.log 2024-05-11T01:16:45.992 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55548.log: /var/log/ceph/ceph-client.admin.48157.log: 89.3% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55548.log.gz 2024-05-11T01:16:45.992 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48157.log.gz 2024-05-11T01:16:45.992 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54076.log 2024-05-11T01:16:45.992 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-05-11T01:16:45.992 INFO:teuthology.orchestra.run.smithi052.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45490.log.gz 2024-05-11T01:16:45.992 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44672.log 2024-05-11T01:16:45.992 INFO:teuthology.orchestra.run.smithi052.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54076.log.gz 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.43166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55632.log 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44672.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44672.log.gz 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41594.log 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44230.log 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55632.log: /var/log/ceph/ceph-client.admin.41594.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55632.log.gz 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41594.log.gz 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46569.log 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46218.log 2024-05-11T01:16:45.993 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.2.44230.log: /var/log/ceph/ceph-client.admin.46569.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46569.log.gz 2024-05-11T01:16:45.994 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-11T01:16:45.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46218.log: 88.8% -- replaced with /var/log/ceph/ceph-client.2.44230.log.gz 2024-05-11T01:16:45.994 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53729.log 2024-05-11T01:16:45.994 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46218.log.gz 2024-05-11T01:16:45.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42956.log 2024-05-11T01:16:45.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53729.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53729.log.gz 2024-05-11T01:16:45.994 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54524.log 2024-05-11T01:16:45.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42956.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42956.log.gz 2024-05-11T01:16:45.994 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49738.log 2024-05-11T01:16:45.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.54524.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54524.log.gz 2024-05-11T01:16:45.995 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43082.log 2024-05-11T01:16:45.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49738.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49738.log.gz 2024-05-11T01:16:45.995 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-05-11T01:16:45.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.43082.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43082.log.gz 2024-05-11T01:16:45.995 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42045.log 2024-05-11T01:16:45.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45658.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-05-11T01:16:45.995 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45406.log 2024-05-11T01:16:45.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42045.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42045.log.gz 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46485.log 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45406.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45406.log.gz 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42711.log 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46485.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46485.log.gz 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49679.log 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42711.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42711.log.gz 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42040.log 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.49679.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49679.log.gz 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51264.log 2024-05-11T01:16:45.996 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42040.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42040.log.gz 2024-05-11T01:16:45.997 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56094.log 2024-05-11T01:16:45.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.51264.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51264.log.gz 2024-05-11T01:16:45.997 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42377.log 2024-05-11T01:16:45.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.56094.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56094.log.gz 2024-05-11T01:16:45.997 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42998.log 2024-05-11T01:16:45.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42377.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42377.log.gz 2024-05-11T01:16:45.997 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41720.log 2024-05-11T01:16:45.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42998.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42998.log.gz 2024-05-11T01:16:45.997 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44630.log 2024-05-11T01:16:45.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41720.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41720.log.gz 2024-05-11T01:16:45.998 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46611.log 2024-05-11T01:16:45.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.44630.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44630.log.gz 2024-05-11T01:16:45.998 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41762.log 2024-05-11T01:16:45.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.46611.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46611.log.gz 2024-05-11T01:16:45.998 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48073.log 2024-05-11T01:16:45.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41762.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41762.log.gz 2024-05-11T01:16:45.998 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53470.log 2024-05-11T01:16:45.998 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.48073.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48073.log.gz 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47634.log 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.53470.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53470.log.gz 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41426.log 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.47634.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47634.log.gz 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45861.log 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41426.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41426.log.gz 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41157.log 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45861.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45861.log.gz 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56303.log 2024-05-11T01:16:45.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.41157.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41157.log.gz 2024-05-11T01:16:46.000 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35748.log 2024-05-11T01:16:46.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.56303.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56303.log.gz 2024-05-11T01:16:46.000 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43040.log 2024-05-11T01:16:46.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.35748.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35748.log.gz 2024-05-11T01:16:46.000 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45448.log 2024-05-11T01:16:46.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.43040.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43040.log.gz 2024-05-11T01:16:46.000 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42716.log 2024-05-11T01:16:46.000 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.45448.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45448.log.gz 2024-05-11T01:16:46.000 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43854.log 2024-05-11T01:16:46.001 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.42716.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42716.log.gz 2024-05-11T01:16:46.001 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55968.log 2024-05-11T01:16:46.001 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.43854.log: 93.9% -- replaced with /var/log/ceph/ceph-client.0.47063.log.gz 2024-05-11T01:16:46.001 INFO:teuthology.orchestra.run.smithi052.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43854.log.gz 2024-05-11T01:16:46.001 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.55968.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55968.log.gz 2024-05-11T01:16:46.179 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.b.log: 92.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-11T01:16:46.193 INFO:teuthology.orchestra.run.smithi119.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-11T01:16:46.397 INFO:teuthology.orchestra.run.smithi119.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-11T01:16:46.407 INFO:teuthology.orchestra.run.smithi119.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-11T01:16:46.465 INFO:teuthology.orchestra.run.smithi119.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-11T01:16:46.470 INFO:teuthology.orchestra.run.smithi052.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-11T01:16:46.612 INFO:teuthology.orchestra.run.smithi052.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-11T01:16:46.642 INFO:teuthology.orchestra.run.smithi119.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-11T01:16:46.669 INFO:teuthology.orchestra.run.smithi052.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-11T01:16:46.850 INFO:teuthology.orchestra.run.smithi052.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-11T01:16:46.954 INFO:teuthology.orchestra.run.smithi052.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-11T01:16:47.016 INFO:teuthology.orchestra.run.smithi119.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-11T01:16:47.018 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-05-11T01:16:47.018 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m1.197s 2024-05-11T01:16:47.018 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m4.328s 2024-05-11T01:16:47.019 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m0.162s 2024-05-11T01:16:47.503 INFO:teuthology.orchestra.run.smithi052.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-11T01:16:47.505 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-11T01:16:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:real 0m1.706s 2024-05-11T01:16:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:user 0m6.002s 2024-05-11T01:16:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:sys 0m0.363s 2024-05-11T01:16:47.506 INFO:tasks.ceph:Archiving logs... 2024-05-11T01:16:47.506 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/remote/smithi052/log 2024-05-11T01:16:47.507 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T01:16:48.397 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/remote/smithi119/log 2024-05-11T01:16:48.398 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T01:16:48.918 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-11T01:16:48.927 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-11T01:16:48.928 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T01:16:48.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T01:16:49.011 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-11T01:16:49.021 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-11T01:16:49.021 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T01:16:49.024 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T01:16:49.038 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-05-11T01:16:49.041 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-05-11T01:16:49.043 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T01:16:49.043 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-05-11T01:16:49.043 INFO:teuthology.orchestra.run.smithi052.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +1049us[+1010us] +/- 68ms 2024-05-11T01:16:49.043 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -576us[ -615us] +/- 73ms 2024-05-11T01:16:49.043 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 -505us[ -544us] +/- 61ms 2024-05-11T01:16:49.043 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T01:16:49.045 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T01:16:49.045 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-05-11T01:16:49.046 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +1040us[+1011us] +/- 68ms 2024-05-11T01:16:49.046 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -579us[ -609us] +/- 73ms 2024-05-11T01:16:49.046 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 65 -525us[ -555us] +/- 61ms 2024-05-11T01:16:49.046 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T01:16:49.047 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-11T01:16:49.056 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-11T01:16:49.057 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-11T01:16:49.092 DEBUG:teuthology.orchestra.run.smithi052:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T01:16:49.167 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T01:16:49.236 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T01:16:49.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T01:16:49.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T01:16:49.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T01:16:49.297 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-11T01:16:49.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-11T01:16:49.382 INFO:teuthology.task.internal:Duration was 1351.942418 seconds 2024-05-11T01:16:49.382 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-11T01:16:49.447 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-11T01:16:49.448 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T01:16:49.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T01:16:49.491 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T01:16:49.493 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T01:16:50.017 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-11T01:16:50.018 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-05-11T01:16:50.018 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T01:16:50.045 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-05-11T01:16:50.045 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T01:16:50.084 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-11T01:16:50.084 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T01:16:50.088 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T01:16:50.156 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-11T01:16:50.156 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T01:16:50.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T01:16:50.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-11T01:16:50.316 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-11T01:16:50.316 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T01:16:50.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T01:16:50.368 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-11T01:16:50.378 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T01:16:50.386 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T01:16:50.408 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-05-11T01:16:50.432 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-05-11T01:16:50.445 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T01:16:50.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T01:16:50.477 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T01:16:50.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T01:16:50.500 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-11T01:16:50.511 INFO:teuthology.task.internal:Transferring archived files... 2024-05-11T01:16:50.511 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/remote/smithi052 2024-05-11T01:16:50.512 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T01:16:50.553 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038/remote/smithi119 2024-05-11T01:16:50.553 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T01:16:50.586 INFO:teuthology.task.internal:Removing archive directory... 2024-05-11T01:16:50.586 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T01:16:50.590 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T01:16:50.636 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-11T01:16:50.648 INFO:teuthology.task.internal:Not uploading archives. 2024-05-11T01:16:50.648 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-11T01:16:50.698 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-11T01:16:50.698 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T01:16:50.701 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T01:16:50.714 INFO:teuthology.orchestra.run.smithi052.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 01:16 /home/ubuntu/cephtest 2024-05-11T01:16:50.716 INFO:teuthology.orchestra.run.smithi119.stdout: 266036 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 01:16 /home/ubuntu/cephtest 2024-05-11T01:16:50.717 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-11T01:16:50.727 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-11T01:16:50.889 INFO:teuthology.nuke:Checking targets against current locks 2024-05-11T01:16:50.918 DEBUG:teuthology.nuke:shortname: smithi052 2024-05-11T01:16:50.918 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-11T01:16:50.941 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 00:45:54.673314', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNPzuCqcuBbnXZ3bZTAStOBBelgDv+9sQKjLwg+nqyO++1tBJi4TrMejaJOMWTNX7ndUGjaVSrhLo+v9nTXsVkQ='} 2024-05-11T01:16:50.944 DEBUG:teuthology.nuke:shortname: smithi119 2024-05-11T01:16:50.944 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-11T01:16:50.964 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-05-11_00:39:26-fs-wip-mchangir-mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds-distro-default-smithi/7702038', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 00:45:54.674795', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOjac69GAdpZVi+7u90HWwTwSnpInKp6GsFxkoCpGY07c3VhdYd+sDz2vnYoZ+hNKkf3QRjP23ZX33X8quJrn6Q='} 2024-05-11T01:16:50.997 INFO:teuthology.orchestra.console.smithi052:Power off 2024-05-11T01:16:50.998 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-11T01:16:51.012 INFO:teuthology.orchestra.console.smithi119:Power off 2024-05-11T01:16:51.013 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-11T01:16:51.021 DEBUG:teuthology.orchestra.console.smithi052:power off output: Chassis Power Control: Down/Off 2024-05-11T01:16:51.021 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-11T01:16:51.037 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2024-05-11T01:16:51.037 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-11T01:16:51.046 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-05-11T01:16:51.059 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-05-11T01:16:55.047 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-11T01:16:55.060 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-11T01:16:55.157 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-05-11T01:16:55.169 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-05-11T01:16:59.158 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-11T01:16:59.171 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-11T01:16:59.268 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off 2024-05-11T01:16:59.282 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2024-05-11T01:16:59.370 INFO:teuthology.orchestra.console.smithi052:Power off completed 2024-05-11T01:16:59.384 INFO:teuthology.orchestra.console.smithi119:Power off completed 2024-05-11T01:16:59.488 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/snap-schedule} duration: 1351.9424176216125 failure_reason: 'Test failure: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=483982b109f5420ab2c1fbc42157ebf3 status: fail success: false 2024-05-11T01:16:59.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T01:16:59.574 INFO:teuthology.run:FAIL